We have been using ParDo for now as well. And it works fine. Our use case revolves around batching data — a typical case is outputting data to a rate-limited API. We can do that already with existing primitives but it would be nice to have helper functions and an interface for that. Totally understand about priority though :).
Kevin On Fri, 4 Mar 2016 at 11:39 Dan Halperin <[email protected]> wrote: > Hi Kevin, > > Can you say more about your needs for an UnboundedSink API? > > Basically, we have found most Sinks are fairly simple and can be > implemented with just ParDo; so the Unbounded Sink API has generally been > dropping in priority relative to all the work we've got to do for general > Beam-readiness and enabling other Runners. > > I'd love to have a discussion about what you'd like to see in the Sink API. > > Thanks, > Dan > > On Fri, Mar 4, 2016 at 7:59 AM, Jean-Baptiste Onofré <[email protected]> > wrote: > > > Hi Kevin, > > > > it's part of the area where we are working for the IO, yes. > > > > Right now, I'm working on the unbounded source (Java), but next one is > > unbounded sink (we will discuss with Dan about that). > > > > Regards > > JB > > > > > > On 03/04/2016 03:26 PM, Kevin Sookocheff wrote: > > > >> One thing we have been waiting on as Dataflow users is the finalization > of > >> the unbounded sink API. I'm not sure if this would fit into the scope > for > >> this example proposal though. > >> On Fri, Mar 4, 2016 at 4:13 AM Jean-Baptiste Onofré <[email protected]> > >> wrote: > >> > >> Hi guys, > >>> > >>> all good things have an end: I'm back from vacation ;) > >>> > >>> This morning, I'm resuming my work on new IOs and transforms, data > >>> integration oriented. Dan already provided great feedbacks on the JMS > >>> one, and I'm updating it to use "new" Bounded and Unbounded interfaces. > >>> Unfortunately, right now, we don't have a lot of samples of unbounded > IO > >>> in Java. > >>> > >>> Actually, I think it would be great if we provide straight forward > >>> recipes, showing how to implement pipelines using IOs & transform to > >>> solve simple but real problems. The WordCount sample is a good one. We > >>> can provide new one to show IOs/data integration IOs, more complex > >>> transforms, etc. > >>> > >>> Thoughts ? > >>> > >>> Regards > >>> JB > >>> -- > >>> Jean-Baptiste Onofré > >>> [email protected] > >>> http://blog.nanthrax.net > >>> Talend - http://www.talend.com > >>> > >>> > >> > > -- > > Jean-Baptiste Onofré > > [email protected] > > http://blog.nanthrax.net > > Talend - http://www.talend.com > > >
