Flink Jobs disappers

2017-07-06 Thread G.S.Vijay Raajaa
HI, I am using Flink Task manager and Job Manager as docker containers. Strangely, I find the jobs to disappear from the web portal after some time. The jobs don't move to the failed state either. Any pointers will be really helpful. Not able to get a clue from the logs. Kindly let me know if I

Re: how to get topic names in SinkFunction when using FlinkKafkaConsumer010 with multiple topics

2017-07-06 Thread Richard Xin
Thanks,I'm not sure I understand this, what I need is to have single a process subscribing multiple kafka topics, and have a switch clause for different topics in my SinkFunction, did you I need to change the way how the kafka producer to produce the message? Any pointer to code samples will be

Can AsyncFunction be applied to connected streams

2017-07-06 Thread PedroMrChaves
Hello, Is there a way to apply the AsyncFunction to connected streams like in a CoFlatMap? I would like to connect streams from different types and process one of them based on the state created by the other in an asynchronous fashion. Regards, Pedro Chaves - Best Regards, Pedro Chaves --

Re: Java 8 lambdas for CEP patterns won't compile

2017-07-06 Thread David Koch
Thank you! Hoping to see Lambda support added back in soon as well. Regards, David On Mon, Jun 12, 2017 at 1:57 PM, Kostas Kloudas wrote: > Done. > > On Jun 12, 2017, at 12:24 PM, Ted Yu wrote: > > Can you add link to this thread in the JIRA

Referencing Global Window across flink jobs

2017-07-06 Thread G.S.Vijay Raajaa
HI, I have a use case were I need to build a global window with custom trigger. I would like to reference this window across my flink jobs. Is there a possibility that the global window can be referenced? Regards, Vijay Raajaa GS

Re: Registering custom metrics does not work

2017-07-06 Thread Chesnay Schepler
How are you verifying whether it is registered? For the sake of covering all angles: Are you certain that createPartitionIndex is called? On 06.07.2017 08:51, wyphao.2007 wrote: Hi Chesnay, thank you for your reply The code above does not get registered at all. 在2017年07月06 14时45分,

Re:Re: Registering custom metrics does not work

2017-07-06 Thread wyphao.2007
Hi Chesnay, thank you for your reply The code above does not get registered at all. 在2017年07月06 14时45分, "Chesnay Schepler"写道: Hello, Plase provide more information as to how it is not working as expected. Does it throw an

Re: Registering custom metrics does not work

2017-07-06 Thread Chesnay Schepler
Hello, Plase provide more information as to how it is not working as expected. Does it throw an exception, log a warning, is the metric not get registered at all or does the value not changing? On 06.07.2017 08:10, wyphao.2007 wrote: Hi, all I want to know element's latency before write to

Registering custom metrics does not work

2017-07-06 Thread wyphao.2007
Hi, all I want to know element's latency before write to Elasticsearch, so I registering a custom metrics as follow: class CustomElasticsearchSinkFunction extends ElasticsearchSinkFunction[EventEntry] { private var metricGroup: Option[MetricGroup] = None private var latency: Long = _