I believe the contract deliberately doesn't specify that level of detail. It could be called with some very large lazy data structure or it could be called with a short list (or empty list) in memory. The idea is to give as much control to the runner as possible.
Kenn On Mon, Mar 17, 2025 at 8:13 AM Joey Tran <joey.t...@schrodinger.com> wrote: > Trying the dev list instead. > > I can't find any mention in the documentation whether `merge_accumulators` > is called with all the accumulators at once or with an iterator. Perusing > some of the combinefns in the python sdk, it looks like all of them work > with an iterator and don't try to iterate over it twice. > > I'm considering implementing CombineFn execution with my runner such > that only two accumulators are ever loaded into memory at a time. Not sure > if CombineFn contract supports this though. > > On Fri, Mar 14, 2025 at 8:20 AM Joey Tran <joey.t...@schrodinger.com> > wrote: > >> My intuition says no says all the accumulators will just grouped together >> into a single key/accumulators element and that entire element will get >> loaded into memory, but not sure if there's any kind of special magic to >> handle this >> >> On Tue, Mar 11, 2025 at 11:16 PM Joey Tran <joey.t...@schrodinger.com> >> wrote: >> >>> Hey all, >>> >>> Just wondering if CombineFn.merge_accumulators is called with a lazy >>> iterator of accumulators or if it's expected to be called with all the >>> accumulators in memory >>> >>> Best, >>> Joey >>> >>
