Hi Ufuk, I'm a Product Manager on Dataflow at Google. I'll reach out to you personally to answer your more involved Dataflow specific questions.
On Wed, Jan 11, 2017 at 11:12 AM Jean-Baptiste Onofré <[email protected]> wrote: > Hi Ufuk, > > Your questions seem more related to Dataflow (or at least runner). I let > the Dataflow experts answer ;) > > For the slack channel, I can send you an invite. > > Regards > JB > > On 01/11/2017 07:33 PM, ufuk altinok wrote: > > Hey all, > > > > First of all congratulations on being a top-level project now. > > > > I’m very excited about the future of this project, it hugely simplifies > everything around data. > > > > Here’s some questions: > > > > - I always jump and use cutting edge software on production when I trust > the developers enough and this is what I did couple days ago when 0.4 is > released. I don’t mind breaking changes and API stability but still I feel > like I’m walking on a high edge. I’m using in GCP dataflow and so far so > good. However do you have any problems that you foresee I may bump into, my > data is safely stored so in case of hard failures I can recover easily. > > > > - I didn’t wait for Dataflow 2.0 beta release and jumped into beam 0.4, > do you think I should go and use 2.0 now instead of 0.4 as I will be only > using Google’s managed dataflow service. > > > > - How can I try ‘TemplatingDataflowPipelineRunner’ on 0.4 or 2.0? I > think this will be renamed to ‘TemplatingPipelineRunner’? Do you have any > guidance? > Instructions for this using 2.0beta is available here <https://cloud.google.com/dataflow/release-notes/release-notes-java-2#runners> > > > > - You apparently have a Slack channel but I was unable to join cause it > requires invitation, do you invite people outside of the team? > > > > > > These are my questions so far. I will also add some feat requests for > Google Dataflow part, this may not be correct place to post those, if so > ignore those please: > > > > - Ability to delete already finished/failed jobs from list > > - Ability to pause a streaming job > > - Ability to clear job logs, I have a streaming job which had failures > due to a service it connects, particularly JDBC connection failed for > couple minutes and I want to be able to clear those errors so I can see > newer ones and clear the red notification :) > > > > > > > > Again thanks for this great project > > > > Ufuk > > > > -- > Jean-Baptiste Onofré > [email protected] > http://blog.nanthrax.net > Talend - http://www.talend.com >
