Stream Iterations
Hi All, I have a use case for which I need some suggestions. It's a streaming application with kafka source and then groupBy, keyBy and perform some calculations. The output of each calculation has to be a side input for the next calculation and also it needs to be sent to a sink. Right now I'm achieving this by storing the result state in memory and also save it in redis cache. I was looking at delta iterations in flink documentation. It would great if someone can help me understand if I can achieve this using iterations or any other api. Thanks in advance. Regards, Govind
[jira] [Created] (FLINK-5365) Mesos AppMaster/TaskManager should obey sigterm
Eron Wright created FLINK-5365: --- Summary: Mesos AppMaster/TaskManager should obey sigterm Key: FLINK-5365 URL: https://issues.apache.org/jira/browse/FLINK-5365 Project: Flink Issue Type: Bug Components: Cluster Management, Mesos Reporter: Eron Wright Assignee: Eron Wright Priority: Minor The AppMaster and TaskManager are ignoring the sigterm sent by Marathon/Mesos. The reason is simply that the shell scripts used to start them don't pass the signal to java. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-5364) Rework JAAS configuration to support user-supplied entries
Eron Wright created FLINK-5364: --- Summary: Rework JAAS configuration to support user-supplied entries Key: FLINK-5364 URL: https://issues.apache.org/jira/browse/FLINK-5364 Project: Flink Issue Type: Bug Components: Cluster Management Reporter: Eron Wright Assignee: Eron Wright Priority: Critical Recent issues (see linked) have brought to light a critical deficiency in the handling of JAAS configuration. 1. the MapR distribution relies on an explicit JAAS conf, rather than in-memory conf used by stock Hadoop. 2. the ZK/Kafka/Hadoop security configuration is supposed to be independent (one can enable each element separately) but isn't. Perhaps we should rework the JAAS conf code to merge any user-supplied configuration with our defaults, rather than using an all-or-nothing approach. We should also address some recent regressions: 1. The HadoopSecurityContext should be installed regardless of auth mode. For example, verify the use of HADOOP_USER_NAME in 'SIMPLE' auth mode. 2. Fix the use of alternative authentication methods - delegation tokens and Kerberos ticket cache. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-5363) Fire timers when window state is currently empty
Aljoscha Krettek created FLINK-5363:
---
Summary: Fire timers when window state is currently empty
Key: FLINK-5363
URL: https://issues.apache.org/jira/browse/FLINK-5363
Project: Flink
Issue Type: Bug
Components: Streaming
Reporter: Aljoscha Krettek
Assignee: Aljoscha Krettek
Fix For: 1.2.0
Currently, when a {{Trigger}} sets a timer and that timer fires in the future
at a point when there is currently no data in the window state, then that timer
is being ignored.
This is a problem for some users because they manually set cleanup timers and
they need to be called because the trigger needs to cleanup some state. (For
normal time windows this is not a problem, but for special cases built on top
of {{GlobalWindows}} the current behaviour leads to problems.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (FLINK-5362) Implement methods to access BipartiteGraph properties
Ivan Mushketyk created FLINK-5362: - Summary: Implement methods to access BipartiteGraph properties Key: FLINK-5362 URL: https://issues.apache.org/jira/browse/FLINK-5362 Project: Flink Issue Type: Improvement Reporter: Ivan Mushketyk Assignee: Ivan Mushketyk -- This message was sent by Atlassian JIRA (v6.3.4#6332)
