[ https://issues.apache.org/jira/browse/FLINK-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15942407#comment-15942407 ]
ASF GitHub Bot commented on FLINK-5654: --------------------------------------- Github user fhueske commented on the issue: https://github.com/apache/flink/pull/3590 Hi @rtudoran, thanks for doing the benchmark and posting the numbers! The recommended state backend for production settings is the RocksDB backend (see [production-readiness docs](https://ci.apache.org/projects/flink/flink-docs-release-1.2/ops/production_ready.html#choice-of-state-backend)). The in-memory backends store state as objects on the heap and can easily kill the JVM with an OutOfMemoryError. Also the in-memory backends do not de/serialize data, so there is not an actual advantage is using the MapState which was mainly motivated by the reduced serialization effort. There are plans to implement a state backend using managed memory (similar to the batch algorithms). This backend would also serialize and deserialize data to/from pre-allocated byte arrays. So optimizing for de/serialization makes sense, IMO. The `KeyedOneInputStreamOperatorTestHarness` is part of the `flink-streaming-java` test-jar artifact. This is added by adding the following dependency to the `flink-table` `pom.xml`. ```<dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_2.10</artifactId> <version>${project.version}</version> <type>test-jar</type> <scope>test</scope> </dependency> ``` I'll have a detailed look at your PR tomorrow. Best, Fabian > Add processing time OVER RANGE BETWEEN x PRECEDING aggregation to SQL > --------------------------------------------------------------------- > > Key: FLINK-5654 > URL: https://issues.apache.org/jira/browse/FLINK-5654 > Project: Flink > Issue Type: Sub-task > Components: Table API & SQL > Reporter: Fabian Hueske > Assignee: radu > > The goal of this issue is to add support for OVER RANGE aggregations on > processing time streams to the SQL interface. > Queries similar to the following should be supported: > {code} > SELECT > a, > SUM(b) OVER (PARTITION BY c ORDER BY procTime() RANGE BETWEEN INTERVAL '1' > HOUR PRECEDING AND CURRENT ROW) AS sumB, > MIN(b) OVER (PARTITION BY c ORDER BY procTime() RANGE BETWEEN INTERVAL '1' > HOUR PRECEDING AND CURRENT ROW) AS minB > FROM myStream > {code} > The following restrictions should initially apply: > - All OVER clauses in the same SELECT clause must be exactly the same. > - The PARTITION BY clause is optional (no partitioning results in single > threaded execution). > - The ORDER BY clause may only have procTime() as parameter. procTime() is a > parameterless scalar function that just indicates processing time mode. > - UNBOUNDED PRECEDING is not supported (see FLINK-5657) > - FOLLOWING is not supported. > The restrictions will be resolved in follow up issues. If we find that some > of the restrictions are trivial to address, we can add the functionality in > this issue as well. > This issue includes: > - Design of the DataStream operator to compute OVER ROW aggregates > - Translation from Calcite's RelNode representation (LogicalProject with > RexOver expression). -- This message was sent by Atlassian JIRA (v6.3.15#6346)