Hello,

Shen you should probably first check the benchmark implementation at
github, I am not sure you need cycles to implement the yahoo benchmark.
Notice for example that there is a flink based implementation and AFAIK
there is no support for cycles in Flink (or at least there wasn't at the
moment they published the benchmarks).

However if you are implementing the yahoo benchmark on Beam, that would be
a nice scenario to test the runner performance (vs the native
implementations), so it would be nice if you can share this.

Regards,
Ismael

On Tue, Nov 29, 2016 at 7:32 PM, Shen LI <cs.she...@gmail.com> wrote:

> Hi Maria, Bobby,
>
> Thanks for the explanation.
>
> Regards,
>
> Shen
>
> On Tue, Nov 29, 2016 at 12:37 PM, Bobby Evans <ev...@yahoo-inc.com.invalid
> >
> wrote:
>
> > In my experience almost all of the time cycles are bad and cause a lot of
> > debugging problems. Most of the time you can implement what you want by
> > using a windowed join or group by instead.
> > - Bobby
> >
> > On Tuesday, November 29, 2016, 11:06:44 AM CST, María García Herrero
> > <mari...@google.com.INVALID> wrote:Hi Shen,
> >
> > No. Beam pipelines are DAGs:
> > http://beam.incubator.apache.org/documentation/sdks/
> > javadoc/0.3.0-incubating/org/apache/beam/sdk/Pipeline.html
> > Best,
> >
> > María
> >
> > On Tue, Nov 29, 2016 at 7:44 AM, Shen LI <cs.she...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > Can I use Beam to create a pipeline with cycles? For example, to
> > implement
> > > the Yahoo! Streaming benchmark(
> > > https://yahooeng.tumblr.com/post/135321837876/benchmarking-streaming-
> > > computation-engines-at),
> > > can a ParDo transform consume a downstream output as a side input?
> > >
> > > Thanks,
> > >
> > > Shen
> > >
> >
>

Reply via email to