Re: Task and Operator Monitoring via JMX / naming

Some further observations: I had a Job which was taking events of a Kafka topic and sending it to two sinks whereas for one of them a Map operation would happen first. When creating one event stream and sending it to the two sinks the JMX representation was not showing both sinks and the naming of

Feasability Question: Distributed FlinkCEP

Hi all, here's what I want to do: Consider a query such as *(A and B) followed_by (C or D)*. (Pseudo code, obviously.) I want to create a graph of independent processing nodes, each running an independent instance of FlinkCEP. I want each of them to represent an operator of the query above, like

Re: NoClassDefFoundError on cluster with httpclient 4.5.2

Hi Yassine, can you check whether the httpclient jar is contained in your job jar which you submit to the cluster? Cheers, Till On Wed, Oct 19, 2016 at 6:41 PM, Yassine MARZOUGUI < y.marzou...@mindlytix.com> wrote: > Hi all, > > I'm using httpclient with the following dependency: > > >

Re: Task and Operator Monitoring via JMX / naming

This is completely unintended behavior; you should never have to adjust your topology so the metric system get's the names right. I'll take a deep look into this tomorrow ;) Regards, Chesnay On 20.10.2016 08:50, Philipp Bussche wrote: Some further observations: I had a Job which was taking

Re: Task and Operator Monitoring via JMX / naming

Well the issue is the following: the metric system assumes the following naming scheme for tasks based on the DataSet API and simple streaming jobs: [CHAIN] operatorName1 [=> operatorName2 [ ...]] To retrieve the operator name the above is split by "=>", giving us a String[] of all operator

Re: org.apache.flink.core.fs.Path error?

I believe i found the issue. The ContinuousFileMonitoringFunction never converts the given string to a Path, but directly generates a URI from it. On 20.10.2016 15:48, Fabian Hueske wrote: The error message suggests that Flink tries to resolve "D:" as a file system schema such as "file:" or

Re: org.apache.flink.core.fs.Path error?

Hello Radu, Flink can handle windows paths, this alone can't be the problem. If you could post the error you are getting we may pinpoint the issue, but right now i would suggest the usual: check that the path is indeed correct, that you have sufficient permissions to access the file. And

Re: org.apache.flink.core.fs.Path error?

Hey, you can ignore the second error, that one is unrelated. I also just tried out passing absolute paths for different drives and I am not encountering issues. :/ Could you check the return value of the following line? Just add it to your job. new File("D:\\dir\\myfile.csv").exists();

Flink error: Too few memory segments provided

I got this error in Gelly, which is a result of flink (i believe) Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job execution failed. at

question for generic streams

Hi, I am trying to read the types of an input stream from a file and then generate the datastream resulting from parsing a source accordingly (e.g. DataStream>). Finally this will be registered as a table. What would be the best way to do this? I am trying currently to

Re: Task and Operator Monitoring via JMX / naming

It will be in the master tomorrow. On 20.10.2016 18:50, Philipp Bussche wrote: Thanks Chesnay ! I am not too familiar with the release cycles here but was wondering when one could expect your fix to be in the master of Flink ? Should I create my own build for the moment maybe ? Thanks. --

ValueState in RichCoFlatMap, possible 1.2-SNAPSHOT regression

Hi all, I was trying to implement a join similar to what was laid out in the flink forward talk Joining Infinity: Windowless Stream Processing with Flink and I have been running to some issues. I am

RE: org.apache.flink.core.fs.Path error?

Hi, I know that Flink in general supports files also on windows. For example I just tested successfully with relative file paths (e.g. place the file in the local directory and give just the file name then everything is working correctly). However with absolute paths it does not work as per my

Re: Task and Operator Monitoring via JMX / naming

Thanks Chesnay, I am happy to share more around my environment and do additional testing for this. Also I would be happy to help fixing if we see there might be an issue in the code somewhere. In fact I am still trying to get a Hacktoberfest T-Shirt and I am still pull requests short ;) --

Re: Feasability Question: Distributed FlinkCEP

Could you not do separate followedBy and then perform a join on the resulting alert stream. Pattern p1= followedBy(/*1st*/) Pattern p2= followedBy(/*1st*/) DataStream alertStream1 = CEP.pattern(keyedDs, p1) DataStream alertStream2 = CEP.pattern(keyedDs, p2) Then just join the two alertStream's

RE: org.apache.flink.core.fs.Path error?

Hi, As you found the source of the error I am not sure if the outputs that you asked for are needed anymore. Nevertheless, see below new File("D:\\dir\\myfile.csv").exists(); => true FileSystem.getLocalFileSystem().getFileStatus(new Path()).getPath(); => “file:/D:/dir/myfile.csv” Fabian’s