Re: Flink and factories?

Hi, wow, oh that's indeed a nice solution. Your version still threw some errors: > Caused by: org.apache.flink.api.common.InvalidProgramException: Object > factorytest.Config$1@5143c662 not serializable > Caused by: java.io.NotSerializableException: factorytest.factory.PearFactory I fixed

Re: Flink and factories?

The functions are serialized when env.execute() is being executed. The thing is, as i understand it, that your singleton is simply not part of the serialized function, so it doesn't actually matter when the function is serialized. Storing the factory instance in the function shouldn't be too

Re: Flink and factories?

Hi Chesnay, thank you for looking into this! Is there any way to tell Flink to (re)sync the changed classes and/or tell it to distribute the serialized classes at a given point (e.g. first on a env.execute() ) or so? The thing is, that I'm working on a small framework which bases on flink, so

ApacheCon is now less than a month away!

Dear Apache Enthusiast, ApacheCon Sevilla is now less than a month out, and we need your help getting the word out. Please tell your colleagues, your friends, and members of related technical communities, about this event. Rates go up November 3rd, so register today! ApacheCon, and Apache Big

Re: Issue while restarting from SavePoint

Hi Ufuk, Thank you for looking into the issue. Please find your answers below : (1) In detached mode the configuration seems to be not picked up correctly. That should be independent of the savepoints. Can you confirm this? —> I tried starting a new job in detached mode and the job started on the

NoClassDefFoundError on cluster with httpclient 4.5.2

Hi all, I'm using httpclient with the following dependency: org.apache.httpcomponents httpclient 4.5.2 On local mode, the program works correctly, but when executed on the cluster, I get the following exception: java.lang.Exception: The user defined 'open(Configuration)' method in class

Re: native snappy library not available

The Hadoop config of your Hadoop installation which is loaded in SequenceFileWriter.open() needs to be configured to have "io.compression.codecs" set to include "SnappyCodec". This is probably described in the Hadoop documentation. -Max On Wed, Oct 19, 2016 at 6:09 PM,

multiple processing of streams

Is it possible to process the same stream in two different ways? I can’t find anything in the documentation definitively stating this is possible, but nor do I find anything stating it isn’t. My attempt had some unexpected results, which I’ll explain below: Essentially, I have a stream of

Flink and factories?

Hi, I'm currently working with flink for my bachelor thesis and I'm running into some odd issues with flink in regards to factories. I've built a small "proof of concept" and the code can be found here: https://gitlab.tubit.tu-berlin.de/gehaxelt/flink-factorytest The idea is that a

Re: Read Apache Kylin from Apache Flink

Thanks for the guide, Alberto! -Max On Tue, Oct 18, 2016 at 10:20 PM, Till Rohrmann wrote: > Great to see Alberto. Thanks for sharing it with the community :-) > > Cheers, > Till > > On Tue, Oct 18, 2016 at 7:40 PM, Alberto Ramón > wrote: >> >>

Re: First Program with WordCount - Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/api/common/functions/FlatMapFunction

This usually happens when you enable the 'build-jar' profile from within IntelliJ. This profile assumes you have a Flink installation in the class path which is only true if you submit the job to an existing Flink cluster. -Max On Mon, Oct 17, 2016 at 10:50 AM, Stefan Richter

Re: native snappy library not available

Hi Robert, have you tried putting the snappy java jar in Flink's lib folder? When specifying the classpath manually you have to make sure that all distributed components are also started with this classpath. Cheers, Till On Wed, Oct 19, 2016 at 1:07 PM, wrote: > I

Re: Issue while restarting from SavePoint

Hey Anirudh! As you say, this looks like two issues: (1) In detached mode the configuration seems to be not picked up correctly. That should be independent of the savepoints. Can you confirm this? (2) The program was changed in a non-compatible way after the savepoint. Did you change the

Re: Multiclass classification example

Hello Kursat, We don't have a multi class classifier in FlinkML currently. Regards, Theodore -- Sent from a mobile device. May contain autocorrect errors. On Oct 19, 2016 12:33 AM, "Kürşat Kurt" wrote: > Hi; > > > I am trying to learn Flink Ml lib. > > Where can i