Re: Clarification: use of AllWindowedStream.apply() function

I have gone through this post , where Aljoscha explains that /mapping/ on WindowedStream is /not/ allowed. So, I think I haven't asked the question properly. Here is

Re: Unable to use Scala's BeanProperty with classes

Any help will be highly appreciable, am stuck on this one. On Tue, Feb 14, 2017 at 10:47 PM, Nico Kruber wrote: > Hi Adarsh, > thanks for reporting this. It should be fixed eventually. > > @Timo: do you have an idea for a work-around or quick-fix? > > > Regards >

Watermarks per key

Hi, I’m designing a streaming job whose elements need to be windowed by event time across a large set of keys. All elements are read from the same source. Event time progresses independently across keys. Is it possible to assign timestamps, and thus generate independent watermarks, per keyed

Clarification: use of AllWindowedStream.apply() function

I am trying to understand if the AllWindowedStream.apply() function can be used for creating a DataStream of new types. Here is a portion of the code: case class

Re: JavaDoc 404

Hi Robert, Thanks for reporting back! The docs look much better now. Cheers, Yassine On Feb 14, 2017 14:30, "Robert Metzger" wrote: HI Yassine, I've now fixed the javadocs build. It is already rebuild for the 1.2 javadocs:

Re: Missing test dependency in Eclipse with Flink 1.2.0

Great! Thanks a lot On 14 Feb 2017 6:07 p.m., "Nico Kruber" wrote: > I did some digging and this is actually documented in FLINK-4813 [1]. > > To work around this issue, add the following plugin to your build plugins: > > > > org.apache.felix >

Re: Unable to use Scala's BeanProperty with classes

Hi Adarsh, thanks for reporting this. It should be fixed eventually. @Timo: do you have an idea for a work-around or quick-fix? Regards Nico On Tuesday, 14 February 2017 21:11:21 CET Adarsh Jain wrote: > I am getting the same problem when trying to do FlatMap operation on my > POJO class. > >

Re: Missing test dependency in Eclipse with Flink 1.2.0

I did some digging and this is actually documented in FLINK-4813 [1]. To work around this issue, add the following plugin to your build plugins: org.apache.felix maven-bundle-plugin 3.0.1 true true Nico [1]

Re: Missing test dependency in Eclipse with Flink 1.2.0

Hi Nico, thanks for the response. The problem is that I don't use the quickstart example. I have a working set of jobs (in Flink 1.1.4) with some unit tests. In the unit tests I use the following dependency that causes the problem: org.apache.flink

Unable to use Scala's BeanProperty with classes

I am getting the same problem when trying to do FlatMap operation on my POJO class. Exception in thread "main" java.lang.IllegalStateException: Detected more than one setter Am using Flink 1.2, the exception is coming when using FlatMap https://issues.apache.org/jira/browse/FLINK-5070

Re: Inconsistent Results in Event Time based Sliding Window processing

Hmm, without any exceptions in the logs, I'd say that you may be on the right track with elements arriving with timestamps older than the last watermark. You may play around with allowed lateness https://ci.apache.org/projects/flink/flink-docs-release-1.2/dev/ windows.html#allowed-lateness to

Re: Inconsistent Results in Event Time based Sliding Window processing

Hi Nico, Thanks for the reply. There are no exceptions or other errors in the job/task manager logs. I am running this example from Eclipse IDE with Kafka and Zookeeper running separately; in the console there are no errors shown while processing. Previously, we were missing some windows due to

Re: Missing test dependency in Eclipse with Flink 1.2.0

You do not require a plugin, but most probably this dependency was not fetched by Eclipse. Please try a "mvn clean package" in your project and see whether this helps Eclipse. Also, you may create a clean test project with mvn archetype:generate \

Re: JavaDoc 404

HI Yassine, I've now fixed the javadocs build. It is already rebuild for the 1.2 javadocs: https://ci.apache.org/projects/flink/flink-docs-release-1.2/api/java/org/apache/flink/streaming/connectors/fs/bucketing/BucketingSink.html The build for master should be done in 30 minutes. On Wed, Feb 8,

Re: Inconsistent Results in Event Time based Sliding Window processing

Hi Sujit, this does indeed sound strange and we are not aware of any data loss issues. Are there any exceptions or other errors in the job/taskmanager logs? Do you have a minimal working example? Is it that whole windows are not processed or just single items inside a window? Nico On Tuesday,

Missing test dependency in Eclipse with Flink 1.2.0

Hi to all, I've tried to migrate to Flink 1.2.0 and now my Eclipse projects says that they can't find *apacheds-jdbm1* that has packaging bundle. Should I install some plugin? Best, Flavio

Inconsistent Results in Event Time based Sliding Window processing

Hi, I have been using Flink 1.1.1 with Kafka 0.9 to process real time streams. We have written our own Window Function and are processing data with Sliding Windows. We are using Event Time and use a custom watermark generator. We select a particular window out of multiple sliding windows and

Re: A way to control redistribution of operator state?

Hi, I was playing with it more today and I think I've found a workaround. So what I do: 1. I define a constant N logical groups 2. I use consistent hash mapping of data keys to these groups 3. I map these groups to partitions using even distribution (same as Flink distributes state) 4. In a

Re: #kafka/#ssl: how to enable ssl authentication for a new kafka consumer?

Hi Alex, Kafka authentication and data transfer encryption using SSL can be simply done be configuring brokers and the connecting client. You can take a look at this: https://kafka.apache.org/documentation/#security_ssl. The Kafka client that the Flink connector uses can be configured through

Re: A way to control redistribution of operator state?

Hi, there is something that we call "raw keyed“ operator state, which might exactly serve your purpose. It is already used internally by Flink’s window operator, but there exists currently no public API for this feature. Way it works currently is that you obtain input and output streams that