Re: CEP issue

Could you provide some example to reproduce the case? Or the Pattern that you are using? It would help track down the issue. > On 2 Feb 2018, at 13:35, Vishal Santoshi wrote: > > I have pulled in the flink master cep library and the runtime ( the cluster ) > is

Re: Latest version of Kafka

Hi, Flink as for now provides only a connector for Kafka 0.11, which is using KafkaClient in 0.11.x version. However you should be able to use it for reading to/writing from Kafka 1.0 - Kafka claims (and as far as I know it’s true) that Kafka 1.0 is backward compatible with 0.11. Piotrek >

Getting Key from keyBy() in ProcessFunction

Hi, I'm currently using a ProcessFunction after a keyBy() and can't find a way to get the key. I'm currently storing it in a ValueState within processElement and set it all the time, so that I can access it in onTimer(). Is there a better way to get the key? We are using Flink 1.3 at the

Re: CEP issue

This is the pattern. Will create a test case. /** * * @param condition a single condition is applied as a acceptance criteria * @param params defining the bounds of the pattern. * @param the element in the stream * @return compiled pattern alonf with the params. */ public static

Re: Getting Key from keyBy() in ProcessFunction

Hi Jürgen, > On Feb 2, 2018, at 6:24 AM, Jürgen Thomann > wrote: > > Hi, > > I'm currently using a ProcessFunction after a keyBy() and can't find a way to > get the key. Doesn’t your keyBy() take a field (position, or

Flink not writing last few elements to disk

Hi Flink community: I am testing Flink but can't write the final(18 or so elements out to disk) Setup: Using NYC yellow taxi from data 2017-09.csv, I sorted the data on pickup_datetime in bash. I am working in event time. Skeleton program: val ds = senv.readFile(input_format, input_path,

Re: Flink REST API

Hi Raja, The registered tracking URL of the YARN application can be used to issue HTTP requests against the REST API. You can retrieve the URL by using the YARN client: yarn application -list In the output, the rightmost column shows the URL, e.g., Application-Id ...

Reduce parallelism without network transfer.

Hi, Assuming that I have a streaming job, using 30 task managers with 4 slot each. I want to change the parallelism of 1 operator from 120 to 30. Are there anyway so that each subtask of this operator get data from 4 upstream subtasks running in the same task manager, thus avoiding network

Re: RocksDB / checkpoint questions

⁣Sent from TypeApp ​ On Feb 3, 2018, 10:48, at 10:48, Kien Truong wrote: >Hi, >Speaking from my experience, if the distributed disk fail, the >checkpoint will fail as well, but the job will continue running. The >checkpoint scheduler will keep running, so the first

Re: How to enable “upsert mode” for dynamic tables?

Thank you again for your detailed response, Fabian. I have this working now!  I have a follow up question that I'd love your input on, also posted on StackOverflow: https://stackoverflow.com/questions/48591790/apache-flink-best-way-to-architect-many-to-one-join-on-dynamic-table Again, I'm

Re: Extending Flink Slots when running on Yarn

Hi Julio, When you start the Flink YARN session, you have to specify the number of TaskManagers and number of slots per TaskManager. There is currently no officially supported way to add more TaskManagers to a long running YARN session. We are aware of this limitation, and there are ongoing