Re: Assigning operators to slots

The execution within the IDE is most likely not loading the flink-conf.yaml file to read the configuration. When run from the IDE you get a LocalStreamEnvironment, which starts a LocalFlinkMiniCluster. LocalStreamEnvironment is created by StreamExecutionEnvironment.createLocalEnvironment without

Re: Can't start cluster

I followed the guide to install flink on the cluster but no one target folder appears in /flink-dist > giordano@giordano-2-2-100-1:~/flink-1.3.2/flink-dist$ ls > pom.xml src Actually there are no results finding target folder > giordano@giordano-2-2-100-1:~$ find . -name target >

Re: Queryable State

Hi, I am sure I have provided the right job manager details because the connection timeout ip is the task manager where the state is kept. I guess the client is able to reach the job manager and figure out where the state is. Also if I provide a wrong state name, I'm receiving unknown state

Re: Flink flick cancel vs stop

Anyone? On Mon, Sep 11, 2017 at 6:17 PM, Elias Levy wrote: > I was wondering about the status of the flink stop command. At first > blush it would seem as the preferable way to shutdown a Flink job, but it > depends on StoppableFunction being implemented by sources

Re: Can't start cluster

It appears that you're building Flink from source, and attempting to start the cluster from the 'flink-dist/src/...' directory. Please use 'flink-dist/target/...' instead since that's where the built distribution is located. For example, check:

Re: QueryableState - No KvStateLocation found for KvState instance

Hi, As Biplob said this means that the JM cannot find the requested state. The reasons can be one of the above but given that you said you are using the FlinkMiniCluster, I assume you are testing. In this case, it can also be that you start querying your state to soon after the job is submitted,

Re: QueryableState - No KvStateLocation found for KvState instance

Hi Hayden, >From what I know, "No KvStateLocation found for KvState instance with name 'word_sums'" is exactly what it means. Your current job can't find the KVState instance. This could result due to a few reasons that I know of: 1. The jobID you supplied for the queryclient job is not equal to

Re: Can't start cluster

I printed also /flink-bin/bin folder: > root@giordano-2-2-100-1:~/flink-1.3.2/flink-dist/src/main/flink-bin/bin# > ls > config.sh flink-console.sh jobmanager.sh start-cluster.sh > start-zookeeper-quorum.sh stop-zookeeper-quorum.sh flink flink-daemon.sh pyflink.batstart-local.bat

Can't start cluster

Hi, I'm trying to deploy on a flink cluster the jar of my program. Unfortunately I have a problem when I call on the first node: > root@giordano-2-2-100-1:~# sudo > ./flink-1.3.2/flink-dist/src/main/flink-bin/bin/start-cluster.sh > Starting cluster. > find:

Testing recoverable job state

I'm a newbie to Flink and am trying to understand how the recovery works using state backends. I've read the documentation and am now trying to run a simple test to demonstrate the abilities - I'd like to test the recovery of a flink job and how the state is recovered from where it left off

bulk iteration

Hello all, even though I asked this question before, I lost my emails. So I have to ask again. Using Bulk iteration, is there any way to know the number of iterations? Cheers, Alieh

RE: Queryable State

I can see the job running in the FlinkUI for the job, and specifically specified the port for the Job Manager. When I provided a different port, I got an akka exception. Here, it seems that the code is getting further. I think that it might be connected with how I am creating the

Re: BucketingSink never closed

I've just looked at Robert presentation at FF [1] and that's exactly what I was waiting for streaming planning/training... Very useful ;) [1] https://www.youtube.com/watch?v=8l8dCKMMWkw On Wed, Sep 13, 2017 at 12:04 PM, Flavio Pompermaier wrote: > Hi Gordon, > thanks for

Re: Queryable State

Hi, are you sure your jobmanager is running and is accessible from the supplied hostname and port? If you can start up the FLink UI of the job which creates your queryable state, it should have the details of the job manager and the port to be used in this queryable client job. -- Sent from:

Re: Using latency markers

Hi Aitozi, Yes, I think we haven’t really pin-pointed out the actual cause of the problem, but if you have a fix for that and can provide a PR we can definitely look at it! That would be helpful. Before opening a PR, also make sure to first open a JIRA for the issue (I don’t think there is one

Re: Using latency markers

Hi, Aljoscha, the dashboard shown NAN is just because the value of the latencyGague is not numerical， so it can't be shown in dashboard, i removed the other latencydescprition except the sink, so i can see the latency in dashboard, do i need to post a pr? thanks, Aitozi -- Sent from:

Re: LatencyMarker

Hi， I have looked into the link you posted, the dashboard shown NAN is just because the value of the latencyGague is not numerical， so it can't be shown in dashboard. another day i will post a pr about this . Now i have a another pr on the issue: https://issues.apache.org/jira/browse/FLINK-7611.

Re: BucketingSink never closed

Hi Gordon, thanks for your feedback. The main problem for me is that moving from batch to stream should be much easier IMHO. Rows should be a first class citizen in Flink and should be VERY easy to read/write them, while at the moment it seems that Tuples are the dominating type...I don't want to

Re: LatencyMarker

Hi, So far I don't have any updates but I still want to look at this. Best, Aljoscha > On 13. Sep 2017, at 11:46, Tzu-Li (Gordon) Tai wrote: > > Hi, > > There is actually latency metrics in the Web UI, but I think there was also > some previously reported problem on that

Re: LatencyMarker

Hi, There is actually latency metrics in the Web UI, but I think there was also some previously reported problem on that [1]. Is there a JIRA for the patch work you did that you mentioned? If you have a fix for that we could definitely have a look at it. Looping in Aljoscha also, who I think

Re: Exception when using keyby operator

Following up: here’s the JIRA ticket for improving the POJO data type documentation - https://issues.apache.org/jira/browse/FLINK-7614. - Gordon On 11 September 2017 at 10:31:23 AM, Sridhar Chellappa (flinken...@gmail.com) wrote: That fixed my issue. Thanks. I also agree we need to fix the

QueryableState - No KvStateLocation found for KvState instance

I am trying to use queryable state, and am encountering issues when querying the state from the client. I get the following exception: Exception in thread "main" org.apache.flink.runtime.query.UnknownKvStateLocation: No KvStateLocation found for KvState instance with name 'word_sums'.

Re: BucketingSink never closed

Ah, sorry, one correction. Just realized there’s already some analysis of the BucketingSink closing issue in this mail thread. Please ignore my request for relevant logs :) On 13 September 2017 at 10:56:10 AM, Tzu-Li (Gordon) Tai (tzuli...@apache.org) wrote: Hi Flavio, Let me try to

Re: BucketingSink never closed

Hi Flavio, Let me try to understand / look at some of the problems you have encountered. checkpointing: it's not clear which checkpointing system to use and how to tune/monitor it and avoid OOM exceptions. What do you mean be which "checkpointing system” to use? Do you mean state backends?