[jira] [Commented] (FLINK-5756) When there are many values under the same key in ListState, RocksDBStateBackend performances poor

2017-04-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962465#comment-15962465 ] ASF GitHub Bot commented on FLINK-5756: --- Github user rmetzger commented on the issue:

[GitHub] flink issue #3704: [FLINK-5756] Replace RocksDB dependency with FRocksDB

2017-04-09 Thread rmetzger
Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/3704 @SyinChwunLeo: we'll merge the fix to both master and the release 1.2 branch. But you can start using FrocksDB right away, because its available in maven central. You just have to

[jira] [Updated] (FLINK-6284) Incorrect sorting of completed checkpoints in ZooKeeperCompletedCheckpointStore

2017-04-09 Thread Robert Metzger (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Metzger updated FLINK-6284: -- Component/s: State Backends, Checkpointing > Incorrect sorting of completed checkpoints in >

[jira] [Commented] (FLINK-6196) Support dynamic schema in Table Function

2017-04-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962458#comment-15962458 ] ASF GitHub Bot commented on FLINK-6196: --- Github user clarkyzl commented on a diff in the pull

[GitHub] flink pull request #3623: [FLINK-6196] [table] Support dynamic schema in Tab...

2017-04-09 Thread clarkyzl
Github user clarkyzl commented on a diff in the pull request: https://github.com/apache/flink/pull/3623#discussion_r110577213 --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/table/functions/TableFunction.scala --- @@ -135,8 +135,11 @@ abstract class

[jira] [Created] (FLINK-6284) Incorrect sorting of completed checkpoints in ZooKeeperCompletedCheckpointStore

2017-04-09 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-6284: --- Summary: Incorrect sorting of completed checkpoints in ZooKeeperCompletedCheckpointStore Key: FLINK-6284 URL: https://issues.apache.org/jira/browse/FLINK-6284 Project:

[jira] [Commented] (FLINK-5756) When there are many values under the same key in ListState, RocksDBStateBackend performances poor

2017-04-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962375#comment-15962375 ] ASF GitHub Bot commented on FLINK-5756: --- Github user StefanRRichter commented on the issue:

[GitHub] flink issue #3704: [FLINK-5756] Replace RocksDB dependency with FRocksDB

2017-04-09 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3704 We are using the newer StringAppendTESTOperator over the StringAppendOperator in rocksdbjni, because the merge algorithm used in StringAppendOperator has quadratic complexity in the number of

[jira] [Commented] (FLINK-5756) When there are many values under the same key in ListState, RocksDBStateBackend performances poor

2017-04-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962357#comment-15962357 ] ASF GitHub Bot commented on FLINK-5756: --- Github user SyinChwunLeo commented on the issue:

[GitHub] flink issue #3704: [FLINK-5756] Replace RocksDB dependency with FRocksDB

2017-04-09 Thread SyinChwunLeo
Github user SyinChwunLeo commented on the issue: https://github.com/apache/flink/pull/3704 OK, FRocksDB is only available in master branch or we can also use it in flink-1.2.0? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[jira] [Commented] (FLINK-5756) When there are many values under the same key in ListState, RocksDBStateBackend performances poor

2017-04-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962353#comment-15962353 ] ASF GitHub Bot commented on FLINK-5756: --- Github user StephanEwen commented on the issue:

[jira] [Commented] (FLINK-5756) When there are many values under the same key in ListState, RocksDBStateBackend performances poor

2017-04-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962354#comment-15962354 ] ASF GitHub Bot commented on FLINK-5756: --- Github user WangTaoTheTonic commented on the issue:

[GitHub] flink issue #3704: [FLINK-5756] Replace RocksDB dependency with FRocksDB

2017-04-09 Thread WangTaoTheTonic
Github user WangTaoTheTonic commented on the issue: https://github.com/apache/flink/pull/3704 Could you tell what modifications are done in "FRocksDB" and post the url of source code repository? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] flink issue #3704: [FLINK-5756] Replace RocksDB dependency with FRocksDB

2017-04-09 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/3704 @SyinChwunLeo We will try and contribute the patch to RocksDB and will also soon try and move to a newer RocksDB version, as soon as its Java API works again for the required functions. The

[jira] [Commented] (FLINK-3089) State API Should Support Data Expiration

2017-04-09 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962347#comment-15962347 ] Xiaogang Shi commented on FLINK-3089: - [~aljoscha] In current implementation, each RocksDB timer is

[GitHub] flink issue #3704: [FLINK-5756] Replace RocksDB dependency with FRocksDB

2017-04-09 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3704 Unfortunately, all newer versions of RocksDB are broken for some functions when used through the Java API and will segfault when using merge operators. This is the same version as we

[jira] [Commented] (FLINK-5756) When there are many values under the same key in ListState, RocksDBStateBackend performances poor

2017-04-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962346#comment-15962346 ] ASF GitHub Bot commented on FLINK-5756: --- Github user StefanRRichter commented on the issue:

[jira] [Commented] (FLINK-5756) When there are many values under the same key in ListState, RocksDBStateBackend performances poor

2017-04-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962342#comment-15962342 ] ASF GitHub Bot commented on FLINK-5756: --- Github user SyinChwunLeo commented on the issue:

[GitHub] flink issue #3704: [FLINK-5756] Replace RocksDB dependency with FRocksDB

2017-04-09 Thread SyinChwunLeo
Github user SyinChwunLeo commented on the issue: https://github.com/apache/flink/pull/3704 I notice that RocksDB in 4.11.2 version is dependent on zlib-1.2.8, unfortunately, there is a security leak in zlib-1.2.8. https://bugzilla.redhat.com/show_bug.cgi?id=CVE-2016-9840,and fixed

[GitHub] flink issue #3704: [FLINK-5756] Replace RocksDB dependency with FRocksDB

2017-04-09 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3704 LGTM. +1 for merging this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[jira] [Commented] (FLINK-5756) When there are many values under the same key in ListState, RocksDBStateBackend performances poor

2017-04-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962317#comment-15962317 ] ASF GitHub Bot commented on FLINK-5756: --- Github user StefanRRichter commented on the issue:

[jira] [Commented] (FLINK-5756) When there are many values under the same key in ListState, RocksDBStateBackend performances poor

2017-04-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962314#comment-15962314 ] ASF GitHub Bot commented on FLINK-5756: --- GitHub user rmetzger opened a pull request:

[GitHub] flink pull request #3704: [FLINK-5756] Replace RocksDB dependency with FRock...

2017-04-09 Thread rmetzger
GitHub user rmetzger opened a pull request: https://github.com/apache/flink/pull/3704 [FLINK-5756] Replace RocksDB dependency with FRocksDB @StefanRRichter has created a custom RocksDB release that fixes FLINK-5756. You can merge this pull request into a Git repository by running:

[jira] [Commented] (FLINK-4674) File Descriptors not being released after Completion of Flink Job Run via Flink Web Portal

2017-04-09 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962252#comment-15962252 ] Stephan Ewen commented on FLINK-4674: - Sorry for the delayed reaction. Could you post some more

[GitHub] flink issue #3661: [FLINK-4953] Allow access to "time" in ProcessWindowFunct...

2017-04-09 Thread aljoscha
Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/3661 I'm currently in SF for http://sf.flink-forward.org and quite busy. That's why my responses are quite slow, sorry for that. A quick answer: There is the (slightly hard to find)

[jira] [Commented] (FLINK-4953) Allow access to "time" in ProcessWindowFunction.Context

2017-04-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962178#comment-15962178 ] ASF GitHub Bot commented on FLINK-4953: --- Github user aljoscha commented on the issue:

[jira] [Updated] (FLINK-6105) Properly handle InterruptedException in HadoopInputFormatBase

2017-04-09 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated FLINK-6105: -- Description: When catching InterruptedException, we should throw InterruptedIOException instead of

[jira] [Updated] (FLINK-6130) Consider calling resourceManager#getTerminationFuture() with lock held

2017-04-09 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated FLINK-6130: -- Description: In YarnFlinkApplicationMasterRunner#runApplicationMaster() : {code} synchronized (lock) {

[jira] [Updated] (FLINK-6143) Unprotected access to this.flink in LocalExecutor#endSession()

2017-04-09 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated FLINK-6143: -- Description: {code} public void endSession(JobID jobID) throws Exception { LocalFlinkMiniCluster flink =

[jira] [Created] (FLINK-6283) Enable to clear a given file cache path

2017-04-09 Thread Zohar Mizrahi (JIRA)
Zohar Mizrahi created FLINK-6283: Summary: Enable to clear a given file cache path Key: FLINK-6283 URL: https://issues.apache.org/jira/browse/FLINK-6283 Project: Flink Issue Type:

[jira] [Commented] (FLINK-5005) Publish Scala 2.12 artifacts

2017-04-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962087#comment-15962087 ] ASF GitHub Bot commented on FLINK-5005: --- Github user DieBauer commented on the issue:

[GitHub] flink issue #3703: [FLINK-5005] WIP: publish scala 2.12 artifacts

2017-04-09 Thread DieBauer
Github user DieBauer commented on the issue: https://github.com/apache/flink/pull/3703 I'm running into an issue with asm in for example the flink-scala module, when compiling with 2.12. ``` java.io.IOException: Class not found at

[jira] [Commented] (FLINK-6090) Add RetractionRule at the stage of decoration

2017-04-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962053#comment-15962053 ] ASF GitHub Bot commented on FLINK-6090: --- Github user godfreyhe commented on a diff in the pull

[GitHub] flink pull request #3696: [FLINK-6090] [table] Add RetractionRule at the sta...

2017-04-09 Thread godfreyhe
Github user godfreyhe commented on a diff in the pull request: https://github.com/apache/flink/pull/3696#discussion_r110534557 --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/table/plan/nodes/datastream/DataStreamRetractionRule.scala --- @@ -0,0 +1,341 @@

[jira] [Commented] (FLINK-6090) Add RetractionRule at the stage of decoration

2017-04-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962052#comment-15962052 ] ASF GitHub Bot commented on FLINK-6090: --- Github user godfreyhe commented on a diff in the pull

[GitHub] flink pull request #3696: [FLINK-6090] [table] Add RetractionRule at the sta...

2017-04-09 Thread godfreyhe
Github user godfreyhe commented on a diff in the pull request: https://github.com/apache/flink/pull/3696#discussion_r110534493 --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/table/plan/nodes/datastream/DataStreamRetractionRule.scala --- @@ -0,0 +1,341 @@