Thanks Sid So, any pointer on how one would interact with it. I mean all I do is assemble DAG and I can’t seem to see anything on the Vertex that would allow me to do that.
Thanks Oleg On May 18, 2015, at 2:00 PM, Siddharth Seth <[email protected]<mailto:[email protected]>> wrote: There's APIs on the ProcessorContext - waitForAllInputsReady, waitForAnyInputReady - which can be used to figure out when a specific Input is ready for consumption. That should solve the first question. Regarding vertices with multiple Inputs and Shuffle - that requires a custom VertexManager plugin to figure out how the splits are to be distributed to the various tasks. Also have to make sure that the number of tasks is setup correctly - likely according to the Shuffle edge. On Mon, May 18, 2015 at 9:00 AM, Oleg Zhurakousky <[email protected]<mailto:[email protected]>> wrote: Also, while trying something related to this i’ve noticed the following: "A vertex with an Initial Input and a Shuffle Input are not supported at the moment”. Is there a target timeframe for this? JIRA? Thanks Oleg > On May 18, 2015, at 10:27 AM, Oleg Zhurakousky > <[email protected]<mailto:[email protected]>> wrote: > > Is it possible to allow Tez processor implementation which has multiple > inputs to become available as soon as at least one input is available to be > read. > This could allow for some computation to begin while waiting for other > inputs. Other inputs could (if logic allows) be processed as they become > available. > > > Thanks > Oleg
