Re: [RESULT] [VOTE] Release Apache Flink 1.1.4 (RC1)

If opened a PR [1] for FLINK-2662 (optimizer chooses invalid shipping strategy between union operators). Would be good to include this fix for 1.1.4. Several users reported this issue. Cheers, Fabian [1] https://github.com/apache/flink/pull/2848 2016-11-16 9:42 GMT+01:00 Ufuk Celebi

[jira] [Created] (FLINK-5122) Elasticsearch Sink loses documents when cluster has high load

static-max created FLINK-5122: - Summary: Elasticsearch Sink loses documents when cluster has high load Key: FLINK-5122 URL: https://issues.apache.org/jira/browse/FLINK-5122 Project: Flink Issue

Re: [DISCUSS] Hold copies in HeapStateBackend

Right, but that would be a much bigger change than "just" copying the *first* record that goes into the ReduceState, or am I missing something? 2016-11-21 18:41 GMT+01:00 Aljoscha Krettek : > To bring over my comment from the Github PR that started this discussion: > >

Re: [DISCUSS] Hold copies in HeapStateBackend

To bring over my comment from the Github PR that started this discussion: @wuchong , yes this is a problem with the HeapStateBackend. The RocksDB backend does not suffer from this problem. I think in the long run we should migrate the HeapStateBackend to always keep

[jira] [Created] (FLINK-5120) Show TM PID in web frontend

Ufuk Celebi created FLINK-5120: -- Summary: Show TM PID in web frontend Key: FLINK-5120 URL: https://issues.apache.org/jira/browse/FLINK-5120 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-5119) Last taskmanager heartbeat not showing in web frontend

Ufuk Celebi created FLINK-5119: -- Summary: Last taskmanager heartbeat not showing in web frontend Key: FLINK-5119 URL: https://issues.apache.org/jira/browse/FLINK-5119 Project: Flink Issue Type:

[jira] [Created] (FLINK-5118) Inconsistent records sent/received metrics

Ufuk Celebi created FLINK-5118: -- Summary: Inconsistent records sent/received metrics Key: FLINK-5118 URL: https://issues.apache.org/jira/browse/FLINK-5118 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-5117) Make the state of the AbstractAlignedProcessingTimeWindowOperator compatible with the WindowOperator.

Kostas Kloudas created FLINK-5117: - Summary: Make the state of the AbstractAlignedProcessingTimeWindowOperator compatible with the WindowOperator. Key: FLINK-5117 URL:

[jira] [Created] (FLINK-5116) Missing RocksDB state backend factory

Ufuk Celebi created FLINK-5116: -- Summary: Missing RocksDB state backend factory Key: FLINK-5116 URL: https://issues.apache.org/jira/browse/FLINK-5116 Project: Flink Issue Type: Bug Affects

[jira] [Created] (FLINK-5115) Minimum time between checkpoints computed from trigger timestamp

Ufuk Celebi created FLINK-5115: -- Summary: Minimum time between checkpoints computed from trigger timestamp Key: FLINK-5115 URL: https://issues.apache.org/jira/browse/FLINK-5115 Project: Flink

[jira] [Created] (FLINK-5114) PartitionState update with finished execution fails

Ufuk Celebi created FLINK-5114: -- Summary: PartitionState update with finished execution fails Key: FLINK-5114 URL: https://issues.apache.org/jira/browse/FLINK-5114 Project: Flink Issue Type:

[jira] [Created] (FLINK-5113) Make all the testing Functions implement the CheckpointedFucntion interface.

Kostas Kloudas created FLINK-5113: - Summary: Make all the testing Functions implement the CheckpointedFucntion interface. Key: FLINK-5113 URL: https://issues.apache.org/jira/browse/FLINK-5113

[jira] [Created] (FLINK-5112) Remove unused accumulator code from ArchivedExecutionJobVertex

Till Rohrmann created FLINK-5112: Summary: Remove unused accumulator code from ArchivedExecutionJobVertex Key: FLINK-5112 URL: https://issues.apache.org/jira/browse/FLINK-5112 Project: Flink

Re: [DISCUSS] Hold copies in HeapStateBackend

Another approach that would solve the problem for our use case (object re-usage for incremental window ReduceFunctions) would be to copy the first object that is put into the state. This would be a change on the ReduceState, not on the overall state backend, which should be feasible, no?

Re: [DISCUSS] Hold copies in HeapStateBackend

-1 for copying objects. Storing a serialized data where possible is good, but copying all objects by default is not a good idea, in my opinion. A lot of scenarios use data types that are hellishly expensive to copy. Even the current copy on chain handover is a problem. Let's not introduce even

[jira] [Created] (FLINK-5111) Change the assignToKeyGroup() method to not use Object.hashCode()

Kostas Kloudas created FLINK-5111: - Summary: Change the assignToKeyGroup() method to not use Object.hashCode() Key: FLINK-5111 URL: https://issues.apache.org/jira/browse/FLINK-5111 Project: Flink

Re: [DISCUSS] Hold copies in HeapStateBackend

Hi, it will come with performance overhead when updating the state, but I think it'll be possible to perform asynchronous snapshots using HeapStateBackend (probably some changes to underlying data structures would be needed) - which would bring more predictable performance. thanks, maciek

Re: [DISCUSS] Schedule and Scope for Flink 1.2

Hi to all, we've completed the PR #2790 to implement FLINK-4491 : we've added the comment that @zentol asked for a week ago and we also fixed a potential annoying problem we faced during indexing (if a

[jira] [Created] (FLINK-5110) Remove the AbstractAlignedProcessingTimeWindowOperator.

Kostas Kloudas created FLINK-5110: - Summary: Remove the AbstractAlignedProcessingTimeWindowOperator. Key: FLINK-5110 URL: https://issues.apache.org/jira/browse/FLINK-5110 Project: Flink

Re: [DISCUSS] Hold copies in HeapStateBackend

Hi, I would be in favour of this since it brings things in line with the RocksDB backend. This will, however, come with quite the performance overhead, depending on how fast the TypeSerializer can copy. Cheers, Aljoscha On Mon, 21 Nov 2016 at 11:30 Fabian Hueske wrote: > Hi

[jira] [Created] (FLINK-5109) Invalid Content-Encoding Header in REST API responses

Móger Tibor László created FLINK-5109: - Summary: Invalid Content-Encoding Header in REST API responses Key: FLINK-5109 URL: https://issues.apache.org/jira/browse/FLINK-5109 Project: Flink

[DISCUSS] Hold copies in HeapStateBackend

Hi everybody, when implementing a ReduceFunction for incremental aggregation of SQL / Table API window aggregates we noticed that the HeapStateBackend does not store copies but holds references to the original objects. In case of a SlidingWindow, the same object is referenced from different

[jira] [Created] (FLINK-5108) Remove ClientShutdownHook during job execution

Maximilian Michels created FLINK-5108: - Summary: Remove ClientShutdownHook during job execution Key: FLINK-5108 URL: https://issues.apache.org/jira/browse/FLINK-5108 Project: Flink Issue

[jira] [Created] (FLINK-5107) Job Manager goes out of memory from long history of prior execution attempts

Stefan Richter created FLINK-5107: - Summary: Job Manager goes out of memory from long history of prior execution attempts Key: FLINK-5107 URL: https://issues.apache.org/jira/browse/FLINK-5107

[jira] [Created] (FLINK-5106) improving IncrementalAggregateReduceFunction

sunjincheng created FLINK-5106: -- Summary: improving IncrementalAggregateReduceFunction Key: FLINK-5106 URL: https://issues.apache.org/jira/browse/FLINK-5106 Project: Flink Issue Type:

[jira] [Created] (FLINK-5105) Improve ReduceState: value put into ReducingState should always be a copy

sunjincheng created FLINK-5105: -- Summary: Improve ReduceState: value put into ReducingState should always be a copy Key: FLINK-5105 URL: https://issues.apache.org/jira/browse/FLINK-5105 Project: Flink