Re: Security in Flink

Hi Stephan, Thanks a lot for the explanation. Is there any timeline on when this will be released ? I guess this one will be the important for our case if we want Flink to be deployed in production. Cheers On Tue, Jan 12, 2016 at 6:19 PM, Stephan Ewen wrote: > Hi Sourav! >

eigenvalue solver

Hi, I wanted to know if there are any implementations yet within the Machine Learning Library or generally that can efficiently solve eigenvalue problems in Flink? Or if not do you have suggestions on how to approach a parallel execution maybe with BLAS or Breeze? Thanks in advance! Lydia

Re: eigenvalue solver

Hi Till, Thanks for the Paper Link! Do you have maybe a Code snippet in mind from BLAS, breeze or spark where to Start from? Best regards, Lydia Von meinem iPhone gesendet > Am 12.01.2016 um 10:46 schrieb Till Rohrmann : > > Hi Lydia, > > there is no Eigenvalue solver

Re: eigenvalue solver

Hi Lydia, there is no Eigenvalue solver included in FlinkML yet. But if you want to, then you can give it a try :-) [1] http://www.cs.newpaltz.edu/~lik/publications/Ruixuan-Li-CCPE-2015.pdf Cheers, Till On Tue, Jan 12, 2016 at 9:47 AM, Lydia Ickler wrote: > Hi, > > I

Re: SQL query support in Flink

Hi Sourav, Flink does not support to execute SQL queries on DataSets, yet. We just started an effort to change that. See the discussion on the dev mailing list [1] and the corresponding design document [2]. For simple relational queries, you can use the Table API [3]. Best, Fabian [1]

Exception using flink-connector-elasticsearch

Hi, We are using the sink for ElasticSearch and when we try to run our job we get the following exception: java.lang.ExceptionInInitializerError Caused by: java.lang.IllegalArgumentException: An SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene410' does not exist. You need to

Re: Exception using flink-connector-elasticsearch

Hi Javier, it seems as if you either are missing the lucene-codecs jar in your classpath or that you have a wrong version (not 4.10.4). Could you check in your job jar whether it includes lucence-codecs? If so, could you run mvn dependency:tree in the root directory of your project. There you

Broadcast variable: error in the wiki?

Hi all, I'm starting to get familiar with Flink. With the help of a colleague I've noticed what I think may be an error in the documentation about broadcast variables ( https://cwiki.apache.org/confluence/display/FLINK/Variables+Closures+vs.+Broadcast+Variables ). The documentation states that a

Re: Broadcast variable: error in the wiki?

Hi! Thank you for pointing out the error. I actually fixed it before reading through to the end of your mail where you offered to fix it yourself. If you find more issues, ping me, I can give you edit permissions. Thanks, Stephan On Tue, Jan 12, 2016 at 11:49 AM, Stefano Baghino <

Re: Broadcast variable: error in the wiki?

Hi, Stephan, thank you for the quick reply and for fixing the issue. If I spot more issues I'll gladly contribute to the wiki. On Tue, Jan 12, 2016 at 12:00 PM, Stephan Ewen wrote: > Hi! > > Thank you for pointing out the error. > > I actually fixed it before reading through

Re: Security in Flink

Hi Sourav! If you want to use Flink in a cluster where neither Hadoop/YARN (not soon Mesos) is available, then I assume you have installed Flink in a standalone mode on the cluster already. There is no support in Flink currently to manage user authentication. Few thoughts on how that may evolve

global function over partitions

Hi, I am trying to compute some final statistics over a stream topology. For this I would like to gather all data from all windows and parallel partitions into a single/global window. Could you suggest a solution for this. I saw that the map function has a ".global()" but I end up with the

Re: What port should I choose to run the SocketTextStreamWordCount sample

Thanks. That summed it up nicely. I’m able to move forward now. Prez Cannady e: revp...@enerscore.com p: 617 500 3378 w: http://www.enerscore.com/ d: http://demo2.enerscore.com/#/ > On Jan 12, 2016,

Re: Exception using flink-connector-elasticsearch

Hi, could you please try adding the lucene-core-4.10.4.jar file to your lib folder of Flink. (https://repo1.maven.org/maven2/org/apache/lucene/lucene-core/4.10.4/) Elasticsearch uses dependency injection to resolve the classes and maven is not really aware of this. Also you could try adding

Re: How to sort tuples in DataStream

Hi! Since a stream is infinite, you cannot simply sort it (Flink does not follow the mini batch model). You can only sort in windows. I assume you key by word and sum up the counts. Since you want to get the most frequent words, you would need to sort across keys, which you can do in a

Re: Security in Flink

Hi Welly! In the end, all remote communication in Flink will go through Netty (Flink direct shuffles do, and Akka uses Netty as well). Netty authenticates connections and encrypts data via SSL, implementing Java's SSLContext. As far as I know, the available algorithms for encryption /

Re: eigenvalue solver

I'm sorry, I haven't. But there should be plenty of implementations out there. Cheers, Till On Tue, Jan 12, 2016 at 11:07 AM, Lydia Ickler wrote: > Hi Till, > Thanks for the Paper Link! > Do you have maybe a Code snippet in mind from BLAS, breeze or spark where > to

Re: What port should I choose to run the SocketTextStreamWordCount sample

Hi, I answered your question on Stack Overflow. To summarize, the program reads data from a text socket. You need to open the socket before starting the program. You can open the socket on any (available) port. If you run nc -lk from a command line, you should used localhost and port

What port should I choose to run the SocketTextStreamWordCount sample

Feeling my way around Flink. I've generated the SocketTextStreamWordCount from the quickstart and discovered quickly the recommended usage Usage: SocketTextStreamWordCount However, I have no idea what port to choose. The QuickStart docs don't reveal much, and while I suspect I need to

akka.pattern.AskTimeoutException

Hi, I am having an error while running some Flink transformations in a DataStream Scala API. The error I get is: Timeout while waiting for JobManager answer. Job time exceeded 21474835 seconds ... Caused by: akka.pattern.AskTimeoutException: Ask timed out on