ashb commented on a change in pull request #15034:
URL: https://github.com/apache/airflow/pull/15034#discussion_r602597687
##########
File path: airflow/utils/task_group.py
##########
@@ -95,8 +95,22 @@ def __init__(
self.used_group_ids = self._parent_group.used_group_ids
self._group_id = group_id
- if self.group_id in self.used_group_ids:
- raise DuplicateTaskIdFound(f"group_id '{self.group_id}' has
already been added to the DAG")
+ # if given group_id already used assign suffix by incrementing largest
used suffix integer
+ # Example : task_group ==> task_group__1 -> task_group__2 ->
task_group__3
+ if group_id in self.used_group_ids:
+ base = re.split(r'__\d+$', group_id)[0]
+ suffixes = sorted(
+ [
+ int(re.split(r'^.+__', used_group_id)[1])
+ for used_group_id in self.used_group_ids
+ if used_group_id is not None and
re.match(rf'^{base}__\d+$', used_group_id)
+ ]
+ )
+ if not suffixes:
+ self._group_id += '__1'
+ else:
+ self._group_id = f'{base}__{suffixes[-1] + 1}'
+
Review comment:
Should this only apply to the decorator? IF you aren't using the
decorator function and you try to reuse a TG id that kind of feels like it
should be an error.
(I don't have a strong opinion either way.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
