Re: [DISCUSS] State of the project: Feature roadmap for 2018

Hi Ben, about the "technical roadmap", we have a thread about "Beam 3.x roadmap". It already provides ideas for points 3 & 4. Regards JB On 01/22/2018 09:15 PM, Ben Chambers wrote: > Thanks Davor for starting the state of the project discussions [1]. > > > In this fork of the state of the pro

Re: Google Dataflow jobs stuck analysing the graph

Try: gcloud beta dataflow jobs --project=myproject cancel myjobname gcloud beta dataflow jobs --project=myproject cancel myjobname2 On Mon, Jan 22, 2018 at 1:28 PM, Carlos Alonso wrote: > We have submitted a couple of jobs that seem to have stuck on the graph > analysing step. > > An weird A job

Google Dataflow jobs stuck analysing the graph

We have submitted a couple of jobs that seem to have stuck on the graph analysing step. An weird A job with ID "2018-01-19_03_27_48-15138951989738918594" doesn't exist error appears on top of the Google Dataflow jobs list page and trying to list it using gcloud tool shows them as in Unknown state:

[DISCUSS] State of the project: Feature roadmap for 2018

Thanks Davor for starting the state of the project discussions [1]. In this fork of the state of the project discussion, I’d like to start the discussion of the feature roadmap for 2018 (and beyond). To kick off the discussion, I think the features could be divided into several areas, as follows:

Re: Strata Conference this March 6-8

Hi Everyone, +1 to the BoF. I'd suggest meeting at 3:30 p.m. on 3/7/2018 at the Philz Coffee Holden suggested. It takes about 10min

Re: KafkaIO - ENABLE_AUTO_COMMIT_CONFIG hardcoded to false?

Kamil, The code at line 1153 you are referring to does not correspond to main Kafka consumer used to fetch the records. KafkaIO source creates an internal consumer to fetch

KafkaIO - ENABLE_AUTO_COMMIT_CONFIG hardcoded to false?

Hi guys, I am using apache beam 2.2.0 on prod. Reading from kafka and writing to BigQuery. In your kafkaIO documentation you mention that one can enable auto-commit for offsets by setting ENABLE_AUTO_COMMIT_CONFIG to true. But I noticed in dataflow logs that my consumer config always has this val