There's APIs on the ProcessorContext - waitForAllInputsReady, waitForAnyInputReady - which can be used to figure out when a specific Input is ready for consumption. That should solve the first question.
Regarding vertices with multiple Inputs and Shuffle - that requires a custom VertexManager plugin to figure out how the splits are to be distributed to the various tasks. Also have to make sure that the number of tasks is setup correctly - likely according to the Shuffle edge. On Mon, May 18, 2015 at 9:00 AM, Oleg Zhurakousky < [email protected]> wrote: > Also, while trying something related to this i’ve noticed the following: > "A vertex with an Initial Input and a Shuffle Input are not supported at > the moment”. > Is there a target timeframe for this? JIRA? > > Thanks > Oleg > > > On May 18, 2015, at 10:27 AM, Oleg Zhurakousky < > [email protected]> wrote: > > > > Is it possible to allow Tez processor implementation which has multiple > inputs to become available as soon as at least one input is available to be > read. > > This could allow for some computation to begin while waiting for other > inputs. Other inputs could (if logic allows) be processed as they become > available. > > > > > > Thanks > > Oleg > >
