https://github.com/python/cpython/commit/59e67c284d3e8dcb708e473cfca02b1307c9cd0a
commit: 59e67c284d3e8dcb708e473cfca02b1307c9cd0a
branch: main
author: Lenormand Julien <[email protected]>
committer: encukou <[email protected]>
date: 2026-07-25T01:40:32+02:00
summary:
gh-118150: warn in doc about results asymmetry for difflib junk (GH-153892)
files:
A Misc/NEWS.d/next/Documentation/2026-07-17-22-33-43.gh-issue-118150.m7iFdP.rst
M Doc/library/difflib.rst
diff --git a/Doc/library/difflib.rst b/Doc/library/difflib.rst
index 3ed4768b6a14136..2176f1b49d0f20f 100644
--- a/Doc/library/difflib.rst
+++ b/Doc/library/difflib.rst
@@ -40,13 +40,18 @@ diffs. For comparing directories and files, see also, the
:mod:`filecmp` module.
complicated way on how many elements the sequences have in common; best case
time is linear.
- **Automatic junk heuristic:** :class:`SequenceMatcher` supports a heuristic
that
- automatically treats certain sequence items as junk. The heuristic counts
how many
- times each individual item appears in the sequence. If an item's duplicates
(after
- the first one) account for more than 1% of the sequence and the sequence is
at least
- 200 items long, this item is marked as "popular" and is treated as junk for
- the purpose of sequence matching. This heuristic can be turned off by
setting
- the ``autojunk`` argument to ``False`` when creating the
:class:`SequenceMatcher`.
+ **Junk**: :class:`SequenceMatcher` accepts an ``isjunk`` predicate and an
+ ``autojunk`` flag. Items that are considered as junk will not be considered
+ to find similar content blocks. This can produce better results for humans
+ (typically breaking on whitespace) and faster (because it reduces the number
+ of possible combinations). But it can also cause pathological cases where
+ too many items considered junk cause an unexpectedly large (but correct)
+ diff result.
+ You should consider tuning them or turning them off depending on your data.
+ Moreover, only the second sequence is inspected for junk. This causes the
diff
+ output to not be symmetrical.
+ When ``autojunk=True``, it will consider as junk the items that account for
more
+ than 1% of the sequence, if it is at least 200 items long.
.. versionchanged:: 3.2
Added the *autojunk* parameter.
@@ -558,16 +563,6 @@ The :class:`SequenceMatcher` class has this constructor:
to try :meth:`quick_ratio` or :meth:`real_quick_ratio` first to get an
upper bound.
- .. note::
-
- Caution: The result of a :meth:`ratio` call may depend on the order of
- the arguments. For instance::
-
- >>> SequenceMatcher(None, 'tide', 'diet').ratio()
- 0.25
- >>> SequenceMatcher(None, 'diet', 'tide').ratio()
- 0.5
-
.. method:: quick_ratio()
diff --git
a/Misc/NEWS.d/next/Documentation/2026-07-17-22-33-43.gh-issue-118150.m7iFdP.rst
b/Misc/NEWS.d/next/Documentation/2026-07-17-22-33-43.gh-issue-118150.m7iFdP.rst
new file mode 100644
index 000000000000000..afa6007a46e1745
--- /dev/null
+++
b/Misc/NEWS.d/next/Documentation/2026-07-17-22-33-43.gh-issue-118150.m7iFdP.rst
@@ -0,0 +1,2 @@
+Clarify in the :mod:`difflib` documentation what *junk* actually does, its
+drawbacks, and how to control it.
_______________________________________________
Python-checkins mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3//lists/python-checkins.python.org
Member address: [email protected]
