These APIs are available during execution of the Processor. They're a mechanism to get notified and wait till certain Inputs are ready, or get notified on an Input being ready while another is being processed. There's nothing on the DAG API for this. What are you looking to do ? One thing to note though - the Inputs are not thread safe, and should be consumed from the same thread or with external synchronization.
On Mon, May 18, 2015 at 11:07 AM, Oleg Zhurakousky < [email protected]> wrote: > Thanks Sid > > So, any pointer on how one would interact with it. I mean all I do is > assemble DAG and I can’t seem to see anything on the Vertex that would > allow me to do that. > > Thanks > Oleg > > > > > On May 18, 2015, at 2:00 PM, Siddharth Seth <[email protected]> wrote: > > There's APIs on the ProcessorContext - waitForAllInputsReady, > waitForAnyInputReady - which can be used to figure out when a specific > Input is ready for consumption. That should solve the first question. > > Regarding vertices with multiple Inputs and Shuffle - that requires a > custom VertexManager plugin to figure out how the splits are to be > distributed to the various tasks. Also have to make sure that the number of > tasks is setup correctly - likely according to the Shuffle edge. > > On Mon, May 18, 2015 at 9:00 AM, Oleg Zhurakousky < > [email protected]> wrote: > >> Also, while trying something related to this i’ve noticed the following: >> "A vertex with an Initial Input and a Shuffle Input are not supported at >> the moment”. >> Is there a target timeframe for this? JIRA? >> >> Thanks >> Oleg >> >> > On May 18, 2015, at 10:27 AM, Oleg Zhurakousky < >> [email protected]> wrote: >> > >> > Is it possible to allow Tez processor implementation which has multiple >> inputs to become available as soon as at least one input is available to be >> read. >> > This could allow for some computation to begin while waiting for other >> inputs. Other inputs could (if logic allows) be processed as they become >> available. >> > >> > >> > Thanks >> > Oleg >> >> > >
