+1 (binding) spread > scatter > unfold > partition > shard > batch
Shahar On Thu, Sep 24, 2026 at 10:01 AM Blain David <[email protected]> wrote: > Hi all, > > The discussion thread [1] about the name of the method that distributes an > iterable over a fixed number of mapped task instances brought up several > candidates without converging on one, so I am opening a vote to settle it > before PR #62922 [2] merges. > What the method does > `task.<name>(N).iterate(url=urls)` creates N mapped task instances through > Dynamic Task Mapping, and each instance iterates over its share of `urls` > in a single task instance using Iterable Tasks. Items are distributed round > robin: item `i` goes to task instance `i % N`. N is the number of task > instances, not a chunk length. That is why `batch(size=N)` was found > misleading: it reads like `itertools.batched`, which does the opposite > (chunk length given, number of chunks derived). > Candidates > In the order they came up in the discussion, with the spelling each would > take: > > 1. > batch - `.batch(tasks=N)` "Task Batching" (original name in the AIP) > 2. > spread - `.spread(across=N)` "Task Spreading" (Shahar, Constance; > currently on the AIP page) > 3. > scatter - `.scatter(across=N)` "Task Scattering" (Jarek) > 4. > unfold - `.unfold(into=N)` "Task Unfolding" (Jarek) > 5. > shard - `.shard(into=N)` "Task Sharding" > 6. > partition - `.partition(into=N)` "Task Partitioning" > > The keyword spellings are indicative. The vote is about the verb; the > exact keyword is settled in the PR once the verb is chosen. > Arguments raised so far > > * > `batch` and `shard` are semantically overloaded (Shahar). Naming it after > * > `itertools.batched` is confusing because the semantics are the opposite > (Ash). > * > `shard` and `partition` carry data-engineering connotations that suggest > contiguous > * > ranges as much as `batch` does. Partitions are now an Airflow feature, so > `partition` > would introduce a name clash (Constance). > * > `spread` is hard to misread because of the preposition: "spread across 17" > cannot be > * > read as "17 each" (Constance). > * > Jarek pointed out that the stdlib settled on a positional `n` for > `batched(it, n)`, > * > which is a precedent to match or consciously break if `batch` is kept. > > Regardless of the outcome > > * > The parameter is keyword-only. `spread(17)` is as ambiguous as `batch(17)` > and is rejected. > * > Passing `size=` raises a clear error instead of being accepted silently. > * > The docs explain the round-robin distribution and the reason for it. > * > The AIP-104 page [3], the Task SDK docs and PR #62922 are updated to the > winning name > * > before the PR merges. > > How to vote > Reply to this thread with your preferences ranked from most to least > preferred, for example: > > spread > scatter > batch > > You do not have to rank all six. The result is decided by instant runoff: > the option with the fewest first preferences is eliminated and its votes > move to the next preference on each ballot, until one option holds a > majority. > > Everyone is welcome to vote. Votes from committers and PMC members are > binding. > The vote stays open for at least 72 hours and closes on Monday 28 > September 2026 at 12:00 CEST. I will post the result in this thread. > > [1] https://lists.apache.org/thread/6qlbvvnt76514rstzk447pvs4784wcj2 > [2] https://github.com/apache/airflow/pull/62922 > [3] > https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/421954527/AIP-104+Iterable+Tasks+IT+and+Task+Spreading+TS > > Thanks, > David > > > General (Internal Property) >
