This is an automated email from the ASF dual-hosted git repository.
tvalentyn pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new fbc9db6bae6 [Python] Return every group from Regex.all_matches (#40563)
fbc9db6bae6 is described below
commit fbc9db6bae64f7a7ed846d323a2716e755c96988
Author: raams95 <[email protected]>
AuthorDate: Wed Oct 14 17:56:45 2026 -0500
[Python] Return every group from Regex.all_matches (#40563)
Regex.all_matches built its output from range(m.lastindex + 1), but
lastindex is the index of the last group that matched (or the outermost
group that closed last), not the number of groups. A regex without
groups failed with a TypeError, and nested groups or an unmatched
trailing optional group were dropped from the output. Use the entire
match followed by m.groups(), which has one entry per group, with None
for a group that did not participate, as Java's AllMatches does with
null.
Fixes #40562
Co-authored-by: JanakiRamChittineni
<[email protected]>
---
CHANGES.md | 1 +
sdks/python/apache_beam/transforms/util.py | 6 ++++--
sdks/python/apache_beam/transforms/util_test.py | 16 ++++++++++++++++
3 files changed, 21 insertions(+), 2 deletions(-)
diff --git a/CHANGES.md b/CHANGES.md
index 1848d909db9..f98790bc2e1 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -83,6 +83,7 @@
`valueMapper`, `topicNameMapper`, or providing a `ConnectionFactory` must
update imports from `javax.jms.*` to
`jakarta.jms.*`. The module now requires Java 17 at runtime
([#38729](https://github.com/apache/beam/issues/38729)).
* (Python) `Regex.find_all` with `group=Regex.ALL` now returns every group, as
`(match, group1, group2, ...)`, instead of only group 1, and with
`outputEmpty=False` it drops a match only when the whole match is empty,
matching the `group=0` behavior
([#40399](https://github.com/apache/beam/pull/40399)).
+* (Python) `Regex.all_matches` now returns every group of the regex, instead
of failing with a `TypeError` when the regex has no groups and dropping nested
or unmatched trailing groups
([#40562](https://github.com/apache/beam/issues/40562)).
## Deprecations
diff --git a/sdks/python/apache_beam/transforms/util.py
b/sdks/python/apache_beam/transforms/util.py
index b7885544dbf..c8c7cc86d71 100644
--- a/sdks/python/apache_beam/transforms/util.py
+++ b/sdks/python/apache_beam/transforms/util.py
@@ -2156,7 +2156,9 @@ class Regex(object):
def all_matches(pcoll, regex):
"""
Returns all matches (groups) if zero or more characters at the beginning
- of string match the regular expression.
+ of string match the regular expression. Each output is a list of the
+ entire match followed by every group of the regex, with None for a group
+ that did not participate in the match.
Args:
regex: the regular expression string or (re.compile) pattern.
@@ -2166,7 +2168,7 @@ class Regex(object):
def _process(element):
m = regex.match(element)
if m:
- yield [m.group(ix) for ix in range(m.lastindex + 1)]
+ yield [m.group(), *m.groups()]
return pcoll | FlatMap(_process)
diff --git a/sdks/python/apache_beam/transforms/util_test.py
b/sdks/python/apache_beam/transforms/util_test.py
index bba09245a0f..1e3db695e07 100644
--- a/sdks/python/apache_beam/transforms/util_test.py
+++ b/sdks/python/apache_beam/transforms/util_test.py
@@ -2854,6 +2854,22 @@ class RegexTest(unittest.TestCase):
["z z", "z", "z"]]
assert_that(result, equal_to(expected_result))
+ def test_all_matched_returns_every_group(self):
+ with TestPipeline() as p:
+ pcol = (p | beam.Create(["ab"]))
+ assert_that(
+ pcol | 'nested groups' >> util.Regex.all_matches("((a)(b))"),
+ equal_to([["ab", "ab", "a", "b"]]),
+ label='CheckNestedGroups')
+ assert_that(
+ pcol | 'optional group' >> util.Regex.all_matches("(a)(c)?"),
+ equal_to([["a", "a", None]]),
+ label='CheckOptionalGroup')
+ assert_that(
+ pcol | 'no groups' >> util.Regex.all_matches("a+"),
+ equal_to([["a"]]),
+ label='CheckNoGroups')
+
def test_match_group_kv(self):
with TestPipeline() as p:
result = (
