Re: Apache Flink Operator State as Query Cache

Hi! In general, if you can keep state in Flink, you get better throughput/latency/consistency and have one less system to worry about (external k/v store). State outside means that the Flink processes can be slimmer and need fewer resources and as such recover a bit faster. There are use cases

Re: Cluster installation gives java.lang.NoClassDefFoundError for everything

Hi Camelia, Flink 0.9.X supports Java 6. So this can't be the issue. Out of curiosity, I gave it a spin on a Linux machine with OpenJDK 6. I was able to start the command-line interface, job manager and task managers. java version "1.6.0_36" OpenJDK Runtime Environment (IcedTea6 1.13.8)

Accumulators/Metrics

Hello, I'm interested in exposing metrics from my UDFs. I see FLINK-1501 exposes task manager metrics via a UI; it would be nice to plug into the same MetricRegistry to register my own (ie, gauges). I don't see this exposed via runtime context. This did lead me to discovering the Accumulators

RE: Cluster installation gives java.lang.NoClassDefFoundError for everything - solved

Hello, Thank you very much for your advices, Stephan, Robert and Maximilian. Upgrading to Java 1.7 on the cluster solved the problem indeed. java version "1.7.0_75" OpenJDK Runtime Environment (rhel-2.5.4.0.el6_6-x86_64 u75-b13) OpenJDK 64-Bit Server VM (build 24.75-b04, mixed mode) Best

Re: Cluster installation gives java.lang.NoClassDefFoundError for everything

Is "jar tf /users/camelia/thecluster/flink-0.9.1/lib/flink-dist-0.9.1.jar" listing for example the "org/apache/flink/client/CliFrontend" class? On Wed, Nov 11, 2015 at 12:09 PM, Maximilian Michels wrote: > Hi Camelia, > > Flink 0.9.X supports Java 6. So this can't be the issue.

Re: finite subset of an infinite data stream

I think what you call "union" is a "connected stream" in Flink. Have a look at this example: https://gist.github.com/fhueske/4ea5422edb5820915fa4 It shows how to dynamically update a list of filters by external requests. Maybe that's what you are looking for? On Wed, Nov 11, 2015 at 12:15 PM,

Re: Flink, Kappa and Lambda

The first and 3rd points here aren't very fair -- they apply to all data systems. Systems downstream of your database can lose data in the same way; the database retention policy expires old data, downstream fails, and back to the tapes you must go. Likewise with 3, a bug in any ETL system can

Re: Flink, Kappa and Lambda

Hi Stephan, Thanks for your response. We are trying to justify whether it's enough to use Kappa Architecture with Flink. This more about resiliency and message lost issue etc. The article is worry about message lost even if you are using Kafka. No matter the message queue or broker you rely

RE: Cluster installation gives java.lang.NoClassDefFoundError for everything

Good morning, Thank you Stephan! I keep on testing and in the meantime I'm wondering if the Java version on the cluster may be part of the issue: java version "1.6.0_36" OpenJDK Runtime Environment (IcedTea6 1.13.8) (rhel-1.13.8.1.el6_7-x86_64) OpenJDK 64-Bit Server VM (build 23.25-b01, mixed