Hi, I am assuming it is an operator that is not the input operator that has the lengthy setup function? Any specific reason you don't want to let data in because things don't get committed (like kafka offsets) till all operators process the data. If you are worried about input operators are going to run-away there is back pressure which will put breaks on when the buffers are full.
Thanks On Thu, Apr 20, 2017 at 3:36 PM, Ganelin, Ilya <ilya.gane...@capitalone.com> wrote: > Is there any way to wait for setup to complete prior to allowing data to > flow through the DAG? I have an operator with a lengthy setup function so > I’m just wondering what the best way to handle this is. My best solution at > the moment is to toggle start with an emission rate of zero and toggle it > on once the DAG is launched. > > > > - Ilya Ganelin > > [image: id:image001.png@01D1F7A4.F3D42980] > > ------------------------------ > > The information contained in this e-mail is confidential and/or > proprietary to Capital One and/or its affiliates and may only be used > solely in performance of work or services for Capital One. The information > transmitted herewith is intended only for use by the individual or entity > to which it is addressed. If the reader of this message is not the intended > recipient, you are hereby notified that any review, retransmission, > dissemination, distribution, copying or other use of, or taking of any > action in reliance upon this information is strictly prohibited. If you > have received this communication in error, please contact the sender and > delete the material from your computer. >
