On 2026/09/24 07:01:24 Blain David wrote:
> Hi all,
>
> The discussion thread [1] about the name of the method that distributes an
> iterable over a fixed number of mapped task instances brought up several
> candidates without converging on one, so I am opening a vote to settle it
> before PR #62922 [2] merges.
> What the method does
> `task.<name>(N).iterate(url=urls)` creates N mapped task instances through
> Dynamic Task Mapping, and each instance iterates over its share of `urls` in
> a single task instance using Iterable Tasks. Items are distributed round
> robin: item `i` goes to task instance `i % N`. N is the number of task
> instances, not a chunk length. That is why `batch(size=N)` was found
> misleading: it reads like `itertools.batched`, which does the opposite (chunk
> length given, number of chunks derived).
> Candidates
> In the order they came up in the discussion, with the spelling each would
> take:
>
> 1. batch - `.batch(tasks=N)` "Task Batching" (original name in the AIP)
> 2. spread - `.spread(across=N)` "Task Spreading" (Shahar, Constance;
> currently on the AIP page)
> 3. scatter - `.scatter(across=N)` "Task Scattering" (Jarek)
> 4. unfold - `.unfold(into=N)` "Task Unfolding" (Jarek)
> 5. shard - `.shard(into=N)` "Task Sharding"
> 6. partition - `.partition(into=N)` "Task Partitioning"
>
> The keyword spellings are indicative. The vote is about the verb; the exact
> keyword is settled in the PR once the verb is chosen.
> Arguments raised so far
>
> * `batch` and `shard` are semantically overloaded (Shahar). Naming it after
> * `itertools.batched` is confusing because the semantics are the opposite
> (Ash).
> * `shard` and `partition` carry data-engineering connotations that suggest
> contiguous
> * ranges as much as `batch` does. Partitions are now an Airflow feature, so
> `partition`
> would introduce a name clash (Constance).
> * `spread` is hard to misread because of the preposition: "spread across
> 17" cannot be
> * read as "17 each" (Constance).
> * Jarek pointed out that the stdlib settled on a positional `n` for
> `batched(it, n)`, which is a precedent to match or consciously break if
> `batch` is kept.
>
> Regardless of the outcome
>
> * The parameter is keyword-only. `spread(17)` is as ambiguous as
> `batch(17)` and is rejected.
> * Passing `size=` raises a clear error instead of being accepted silently.
> * The docs explain the round-robin distribution and the reason for it.
> * The AIP-104 page [3], the Task SDK docs and PR #62922 are updated to the
> winning name before the PR merges.
>
> How to vote
> Reply to this thread with your preferences ranked from most to least
> preferred, for example:
>
> spread > scatter > batch
>
> You do not have to rank all six. The result is decided by instant runoff: the
> option with the fewest first preferences is eliminated and its votes move to
> the next preference on each ballot, until one option holds a majority.
>
> Everyone is welcome to vote. Votes from committers and PMC members are
> binding.
> The vote stays open for at least 72 hours and closes on Monday 28 September
> 2026 at 12:00 CEST. I will post the result in this thread.
>
> [1] https://lists.apache.org/thread/6qlbvvnt76514rstzk447pvs4784wcj2
> [2] https://github.com/apache/airflow/pull/62922
> [3]
> https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/421954527/AIP-104+Iterable+Tasks+IT+and+Task+Spreading+TS
>
> Thanks,
> David
>
>
> General (Internal Property)
>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
