+1 (non-binding) on spread.(across=N) I actually don’t think you need to change the name of the feature to Task Spreading though, it’s fine as is. There’s plenty of times where the feature name and the action or verb itself is named something different.
Oh, then: distribute (Ash’s new one) > scatter > parallelize > … with a strong dislike of unfold > shard > batch > partition. You’re doing a great job wrangling cats :) > On Sep 24, 2026, at 3:13 AM, Shahar Epstein <[email protected]> wrote: > +1 (binding) > > spread > scatter > unfold > partition > shard > batch > > > Shahar > > On Thu, Sep 24, 2026 at 10:01 AM Blain David <[email protected]> > wrote: > >> Hi all, >> >> The discussion thread [1] about the name of the method that distributes an >> iterable over a fixed number of mapped task instances brought up several >> candidates without converging on one, so I am opening a vote to settle it >> before PR #62922 [2] merges. >> What the method does >> `task.<name>(N).iterate(url=urls)` creates N mapped task instances through >> Dynamic Task Mapping, and each instance iterates over its share of `urls` >> in a single task instance using Iterable Tasks. Items are distributed round >> robin: item `i` goes to task instance `i % N`. N is the number of task >> instances, not a chunk length. That is why `batch(size=N)` was found >> misleading: it reads like `itertools.batched`, which does the opposite >> (chunk length given, number of chunks derived). >> Candidates >> In the order they came up in the discussion, with the spelling each would >> take: >> >> 1. >> batch - `.batch(tasks=N)` "Task Batching" (original name in the AIP) >> 2. >> spread - `.spread(across=N)` "Task Spreading" (Shahar, Constance; >> currently on the AIP page) >> 3. >> scatter - `.scatter(across=N)` "Task Scattering" (Jarek) >> 4. >> unfold - `.unfold(into=N)` "Task Unfolding" (Jarek) >> 5. >> shard - `.shard(into=N)` "Task Sharding" >> 6. >> partition - `.partition(into=N)` "Task Partitioning" >> >> The keyword spellings are indicative. The vote is about the verb; the >> exact keyword is settled in the PR once the verb is chosen. >> Arguments raised so far >> >> * >> `batch` and `shard` are semantically overloaded (Shahar). Naming it after >> * >> `itertools.batched` is confusing because the semantics are the opposite >> (Ash). >> * >> `shard` and `partition` carry data-engineering connotations that suggest >> contiguous >> * >> ranges as much as `batch` does. Partitions are now an Airflow feature, so >> `partition` >> would introduce a name clash (Constance). >> * >> `spread` is hard to misread because of the preposition: "spread across 17" >> cannot be >> * >> read as "17 each" (Constance). >> * >> Jarek pointed out that the stdlib settled on a positional `n` for >> `batched(it, n)`, >> * >> which is a precedent to match or consciously break if `batch` is kept. >> >> Regardless of the outcome >> >> * >> The parameter is keyword-only. `spread(17)` is as ambiguous as `batch(17)` >> and is rejected. >> * >> Passing `size=` raises a clear error instead of being accepted silently. >> * >> The docs explain the round-robin distribution and the reason for it. >> * >> The AIP-104 page [3], the Task SDK docs and PR #62922 are updated to the >> winning name >> * >> before the PR merges. >> >> How to vote >> Reply to this thread with your preferences ranked from most to least >> preferred, for example: >> >> spread > scatter > batch >> >> You do not have to rank all six. The result is decided by instant runoff: >> the option with the fewest first preferences is eliminated and its votes >> move to the next preference on each ballot, until one option holds a >> majority. >> >> Everyone is welcome to vote. Votes from committers and PMC members are >> binding. >> The vote stays open for at least 72 hours and closes on Monday 28 >> September 2026 at 12:00 CEST. I will post the result in this thread. >> >> [1] https://lists.apache.org/thread/6qlbvvnt76514rstzk447pvs4784wcj2 >> [2] https://github.com/apache/airflow/pull/62922 >> [3] >> https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/421954527/AIP-104+Iterable+Tasks+IT+and+Task+Spreading+TS >> >> Thanks, >> David >> >> >> General (Internal Property) --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
