Hi Ufuk,
Your questions seem more related to Dataflow (or at least runner). I let
the Dataflow experts answer ;)
For the slack channel, I can send you an invite.
Regards
JB
On 01/11/2017 07:33 PM, ufuk altinok wrote:
Hey all,
First of all congratulations on being a top-level project now.
I’m very excited about the future of this project, it hugely simplifies
everything around data.
Here’s some questions:
- I always jump and use cutting edge software on production when I trust the
developers enough and this is what I did couple days ago when 0.4 is released.
I don’t mind breaking changes and API stability but still I feel like I’m
walking on a high edge. I’m using in GCP dataflow and so far so good. However
do you have any problems that you foresee I may bump into, my data is safely
stored so in case of hard failures I can recover easily.
- I didn’t wait for Dataflow 2.0 beta release and jumped into beam 0.4, do you
think I should go and use 2.0 now instead of 0.4 as I will be only using
Google’s managed dataflow service.
- How can I try ‘TemplatingDataflowPipelineRunner’ on 0.4 or 2.0? I think this
will be renamed to ‘TemplatingPipelineRunner’? Do you have any guidance?
- You apparently have a Slack channel but I was unable to join cause it
requires invitation, do you invite people outside of the team?
These are my questions so far. I will also add some feat requests for Google
Dataflow part, this may not be correct place to post those, if so ignore those
please:
- Ability to delete already finished/failed jobs from list
- Ability to pause a streaming job
- Ability to clear job logs, I have a streaming job which had failures due to a
service it connects, particularly JDBC connection failed for couple minutes and
I want to be able to clear those errors so I can see newer ones and clear the
red notification :)
Again thanks for this great project
Ufuk
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
