Re: Flink Metrics

2016-10-17 Thread Till Rohrmann
Hi Govind, I think the DropwizardMeterWrapper implementation is just a reference implementation where it was decided to report the minute rate. You can define your own meter class which allows to configure the rate interval accordingly. Concerning Timers, I think nobody requested this metric so f

[jira] [Created] (FLINK-4840) Introduce an OperatorSystemMetricGroup

2016-10-17 Thread zhuhaifeng (JIRA)
zhuhaifeng created FLINK-4840: - Summary: Introduce an OperatorSystemMetricGroup Key: FLINK-4840 URL: https://issues.apache.org/jira/browse/FLINK-4840 Project: Flink Issue Type: Improvement

Re: [DISCUSS][FLIP-4] Enhance Window Evictor in Flink

2016-10-17 Thread Aljoscha Krettek
Hi Vishnu, what you suggested is spot on! Please go forward with it like this. One small suggestion would be to change Tuple2 to something like TimestampedValue to not rely on tuples because they can be confusing for people who write Scala code because they are not Scala tuples. That's not strictl

[jira] [Created] (FLINK-4841) Suicide TaskManager if Task doesn't react to cancel signal

2016-10-17 Thread Ufuk Celebi (JIRA)
Ufuk Celebi created FLINK-4841: -- Summary: Suicide TaskManager if Task doesn't react to cancel signal Key: FLINK-4841 URL: https://issues.apache.org/jira/browse/FLINK-4841 Project: Flink Issue Ty

[jira] [Created] (FLINK-4842) Introduce test to enforce order of operator / udf lifecycles

2016-10-17 Thread Stefan Richter (JIRA)
Stefan Richter created FLINK-4842: - Summary: Introduce test to enforce order of operator / udf lifecycles Key: FLINK-4842 URL: https://issues.apache.org/jira/browse/FLINK-4842 Project: Flink

[jira] [Created] (FLINK-4843) Introduce Test for FsCheckpointStateOutputStream::getPos

2016-10-17 Thread Stefan Richter (JIRA)
Stefan Richter created FLINK-4843: - Summary: Introduce Test for FsCheckpointStateOutputStream::getPos Key: FLINK-4843 URL: https://issues.apache.org/jira/browse/FLINK-4843 Project: Flink Issu

[jira] [Created] (FLINK-4844) Partitionable Raw Keyed/Operator State

2016-10-17 Thread Stefan Richter (JIRA)
Stefan Richter created FLINK-4844: - Summary: Partitionable Raw Keyed/Operator State Key: FLINK-4844 URL: https://issues.apache.org/jira/browse/FLINK-4844 Project: Flink Issue Type: New Featur

[DISCUSS] Schedule and Scope for Flink 1.2

2016-10-17 Thread Fabian Hueske
Hi everybody, Flink 1.1.0 was released in August and I think it is time to kick off a discussion about the schedule and scope of Flink 1.2.0. == Scope We started to collect features for Flink 1.2.0 in the Flink Release wiki page [1]. I copy the feature list for convenience: - Dynamic Scaling

[jira] [Created] (FLINK-4845) Fix Job Exceptions page

2016-10-17 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-4845: --- Summary: Fix Job Exceptions page Key: FLINK-4845 URL: https://issues.apache.org/jira/browse/FLINK-4845 Project: Flink Issue Type: Bug Compone

Re: [FLINK-3035] Redis as State Backend

2016-10-17 Thread Aljoscha Krettek
Hi, there are two basic ideas for implementing a StateBackend based on Redis: 1. Rely on Redis to keep the state, use nothing else. 2. Use Redis to keep the state and checkpoint to some distributed file system (such as HDFS) when checkpointing The first idea seems unwise because Redis is not a "

[jira] [Created] (FLINK-4846) FlinkML - Pass "env" has an implicit parameter in MLUtils.readLibSVM

2016-10-17 Thread Thomas FOURNIER (JIRA)
Thomas FOURNIER created FLINK-4846: -- Summary: FlinkML - Pass "env" has an implicit parameter in MLUtils.readLibSVM Key: FLINK-4846 URL: https://issues.apache.org/jira/browse/FLINK-4846 Project: Flink

[jira] [Created] (FLINK-4847) Let RpcEndpoint.start/shutDown throw exceptions

2016-10-17 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-4847: Summary: Let RpcEndpoint.start/shutDown throw exceptions Key: FLINK-4847 URL: https://issues.apache.org/jira/browse/FLINK-4847 Project: Flink Issue Type: Sub

Re: [DISCUSS] Schedule and Scope for Flink 1.2

2016-10-17 Thread Stephan Ewen
I think this sounds very reasonable, +1 to the schedule. I would definitely add - FLIP-10 (unify checkpoints and savepoints) - FLIP-7 (metrics in web UI) - FLIP-12 (async request operators) Those should be all safe bets, as they are basically done. Let's see what else is in shape until the

Re: Flink Metrics

2016-10-17 Thread amir bahmanyari
Hi colleagues,Is there a link that described Flink Matrices & provides example on how to utilize it pls?I really appreciate it...Cheers From: Till Rohrmann To: u...@flink.apache.org Cc: dev@flink.apache.org Sent: Monday, October 17, 2016 12:52 AM Subject: Re: Flink Metrics Hi Govin

Re: [DISCUSS][FLIP-4] Enhance Window Evictor in Flink

2016-10-17 Thread Vishnu Viswanath
Hi Aljoscha, Thanks for the response. I did think about creating a new class similar to TimestampedValue as you suggested, but that class looked almost same as the current StreamRecord class. (Both have a timestamp field and a value field). Do you think it is fine to have another class for holdi

[jira] [Created] (FLINK-4848) keystoreFilePath should be checked against null in SSLUtils#createSSLServerContext

2016-10-17 Thread Ted Yu (JIRA)
Ted Yu created FLINK-4848: - Summary: keystoreFilePath should be checked against null in SSLUtils#createSSLServerContext Key: FLINK-4848 URL: https://issues.apache.org/jira/browse/FLINK-4848 Project: Flink

[jira] [Created] (FLINK-4849) keystoreFilePath should be checked against null in SSLUtils#createSSLServerContext

2016-10-17 Thread Ted Yu (JIRA)
Ted Yu created FLINK-4849: - Summary: keystoreFilePath should be checked against null in SSLUtils#createSSLServerContext Key: FLINK-4849 URL: https://issues.apache.org/jira/browse/FLINK-4849 Project: Flink

Re: Exception from in-progress implementation of Python API bulk iterations

2016-10-17 Thread Geoffrey Mon
Your solution works well, many thanks. This solves the exception that I described previously. However, in a different part of the script I come across another problem about reusing data sets. For example, given the script at https://gist.github.com/GEOFBOT/d670f567f8c886572c8715a6058f8b34, I get a