+1 (binding) on spread(across=N) On Thu, Sep 24, 2026 at 12:35 PM David Blain <[email protected]> wrote:
> > > On 2026/09/24 07:01:24 Blain David wrote: > > Hi all, > > > > The discussion thread [1] about the name of the method that distributes > an iterable over a fixed number of mapped task instances brought up several > candidates without converging on one, so I am opening a vote to settle it > before PR #62922 [2] merges. > > What the method does > > `task.<name>(N).iterate(url=urls)` creates N mapped task instances > through Dynamic Task Mapping, and each instance iterates over its share of > `urls` in a single task instance using Iterable Tasks. Items are > distributed round robin: item `i` goes to task instance `i % N`. N is the > number of task instances, not a chunk length. That is why `batch(size=N)` > was found misleading: it reads like `itertools.batched`, which does the > opposite (chunk length given, number of chunks derived). > > Candidates > > In the order they came up in the discussion, with the spelling each > would take: > > > > 1. batch - `.batch(tasks=N)` "Task Batching" (original name in the AIP) > > 2. spread - `.spread(across=N)` "Task Spreading" (Shahar, Constance; > currently on the AIP page) > > 3. scatter - `.scatter(across=N)` "Task Scattering" (Jarek) > > 4. unfold - `.unfold(into=N)` "Task Unfolding" (Jarek) > > 5. shard - `.shard(into=N)` "Task Sharding" > > 6. partition - `.partition(into=N)` "Task Partitioning" > > > > The keyword spellings are indicative. The vote is about the verb; the > exact keyword is settled in the PR once the verb is chosen. > > Arguments raised so far > > > > * `batch` and `shard` are semantically overloaded (Shahar). Naming it > after > > * `itertools.batched` is confusing because the semantics are the > opposite (Ash). > > * `shard` and `partition` carry data-engineering connotations that > suggest contiguous > > * ranges as much as `batch` does. Partitions are now an Airflow > feature, so `partition` > > would introduce a name clash (Constance). > > * `spread` is hard to misread because of the preposition: "spread > across 17" cannot be > > * read as "17 each" (Constance). > > * Jarek pointed out that the stdlib settled on a positional `n` for > `batched(it, n)`, which is a precedent to match or consciously break if > `batch` is kept. > > > > Regardless of the outcome > > > > * The parameter is keyword-only. `spread(17)` is as ambiguous as > `batch(17)` and is rejected. > > * Passing `size=` raises a clear error instead of being accepted > silently. > > * The docs explain the round-robin distribution and the reason for it. > > * The AIP-104 page [3], the Task SDK docs and PR #62922 are updated to > the winning name before the PR merges. > > > > How to vote > > Reply to this thread with your preferences ranked from most to least > preferred, for example: > > > > spread > scatter > batch > > > > You do not have to rank all six. The result is decided by instant > runoff: the option with the fewest first preferences is eliminated and its > votes move to the next preference on each ballot, until one option holds a > majority. > > > > Everyone is welcome to vote. Votes from committers and PMC members are > binding. > > The vote stays open for at least 72 hours and closes on Monday 28 > September 2026 at 12:00 CEST. I will post the result in this thread. > > > > [1] https://lists.apache.org/thread/6qlbvvnt76514rstzk447pvs4784wcj2 > > [2] https://github.com/apache/airflow/pull/62922 > > [3] > https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/421954527/AIP-104+Iterable+Tasks+IT+and+Task+Spreading+TS > > > > Thanks, > > David > > > > > > General (Internal Property) > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
