[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477438#comment-16477438 ] ASF GitHub Bot commented on FLINK-9373: --- Github user sihuazhou commented on the issue:

[GitHub] flink pull request #6024: [FLINK-9301] [e2e test] Add back "not co mini clus...

2018-05-16 Thread azagrebin
GitHub user azagrebin opened a pull request: https://github.com/apache/flink/pull/6024 [FLINK-9301] [e2e test] Add back "not co mini cluster" test with reduced parallelism …ced to 25 parallelism and memory usage logging ## What is the purpose of the change

[jira] [Updated] (FLINK-9384) KafkaAvroTableSource failed to work due to type mismatch

2018-05-16 Thread Jun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Zhang updated FLINK-9384: - Flags: Patch > KafkaAvroTableSource failed to work due to type mismatch >

[jira] [Assigned] (FLINK-9381) BlobServer data for a job is not getting cleaned up at JM

2018-05-16 Thread Till Rohrmann (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Till Rohrmann reassigned FLINK-9381: Assignee: Till Rohrmann > BlobServer data for a job is not getting cleaned up at JM >

[GitHub] flink issue #6020: [FLINK-9373][state] Fix potential data losing for RocksDB...

2018-05-16 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/6020 After double checking with the RocksDB docs, I am afraid that we need to introduce more checks because for example the point out that also after methods like `seek` the iterator an become

[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477549#comment-16477549 ] ASF GitHub Bot commented on FLINK-9373: --- Github user StefanRRichter commented on the issue:

[GitHub] flink issue #6020: [FLINK-9373][state] Fix potential data losing for RocksDB...

2018-05-16 Thread sihuazhou
Github user sihuazhou commented on the issue: https://github.com/apache/flink/pull/6020 Oh My God...Is that means we need to wrap the `RocksIterator` to delegate all it API? ---

[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477554#comment-16477554 ] ASF GitHub Bot commented on FLINK-9373: --- Github user sihuazhou commented on the issue:

[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477556#comment-16477556 ] ASF GitHub Bot commented on FLINK-9373: --- Github user StefanRRichter commented on the issue:

[GitHub] flink issue #6020: [FLINK-9373][state] Fix potential data losing for RocksDB...

2018-05-16 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/6020 It depends, maybe this is already covered currently because we might always do an iteration attempt that checks right after the seek. But in general, this is not very nice and fragile if

[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477560#comment-16477560 ] ASF GitHub Bot commented on FLINK-9373: --- Github user StefanRRichter commented on the issue:

[GitHub] flink issue #6020: [FLINK-9373][state] Fix potential data losing for RocksDB...

2018-05-16 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/6020 Maybe we should ask them on their issue tracker what the best practise is? I cannot remember seeing such checks in their code examples. Have a hard time to believe that this can be true,

[GitHub] flink issue #6020: [FLINK-9373][state] Fix potential data losing for RocksDB...

2018-05-16 Thread sihuazhou
Github user sihuazhou commented on the issue: https://github.com/apache/flink/pull/6020 Agreed! ---

[GitHub] flink issue #6020: [FLINK-9373][state] Fix potential data losing for RocksDB...

2018-05-16 Thread sihuazhou
Github user sihuazhou commented on the issue: https://github.com/apache/flink/pull/6020 @StefanRRichter I had a look at the implementation of the iterators in RocksDB, I found status just return the flag first `_status` as the result without any complex computation, But for some

[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477465#comment-16477465 ] ASF GitHub Bot commented on FLINK-9373: --- Github user sihuazhou commented on the issue:

[GitHub] flink issue #6020: [FLINK-9373][state] Fix potential data losing for RocksDB...

2018-05-16 Thread sihuazhou
Github user sihuazhou commented on the issue: https://github.com/apache/flink/pull/6020 Agree, should be correct first before fast! Could you please have a look at this? I think it's already for a look now~ ---

[GitHub] flink issue #5421: [FLINK-8573] [client] Add more information for printing J...

2018-05-16 Thread azagrebin
Github user azagrebin commented on the issue: https://github.com/apache/flink/pull/5421 @zhangminglei what do you think about creation one more `ProgramInvocationException` overloaded constructor with JobId as parameter? It could always append to the message like `MESSAGE +

[jira] [Commented] (FLINK-9381) BlobServer data for a job is not getting cleaned up at JM

2018-05-16 Thread Till Rohrmann (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477600#comment-16477600 ] Till Rohrmann commented on FLINK-9381: -- I think we don't properly clean up the job resources after

[jira] [Commented] (FLINK-8659) Add migration tests for Broadcast state.

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477648#comment-16477648 ] ASF GitHub Bot commented on FLINK-8659: --- Github user kl0u commented on the issue:

[GitHub] flink issue #5955: [FLINK-8659] Add migration itcases for broadcast state.

2018-05-16 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/5955 Hi @tzulitai ! Thanks for the review. I integrated most of your comments. The only one I left out is the one about merging the checkpointing and the checking. I am not against that. It is just that the

[jira] [Commented] (FLINK-8918) Introduce Runtime Filter Join

2018-05-16 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477433#comment-16477433 ] Sihua Zhou commented on FLINK-8918: --- [~fhueske] related to the saturated problem, I want to add some

[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477459#comment-16477459 ] ASF GitHub Bot commented on FLINK-9373: --- Github user StefanRRichter commented on the issue:

[GitHub] flink issue #6020: [FLINK-9373][state] Fix potential data losing for RocksDB...

2018-05-16 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/6020 It sounds cheap if they just `&` all the flags from the sub iterators. In the end, we can see if there is a performance drop but better be correct first before fast. ---

[jira] [Created] (FLINK-9384) KafkaAvroTableSource failed to work due to type mismatch

2018-05-16 Thread Jun Zhang (JIRA)
Jun Zhang created FLINK-9384: Summary: KafkaAvroTableSource failed to work due to type mismatch Key: FLINK-9384 URL: https://issues.apache.org/jira/browse/FLINK-9384 Project: Flink Issue Type:

[jira] [Updated] (FLINK-9384) KafkaAvroTableSource failed to work due to type mismatch

2018-05-16 Thread Jun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Zhang updated FLINK-9384: - Attachment: flink-9384.patch > KafkaAvroTableSource failed to work due to type mismatch >

[GitHub] flink issue #6020: [FLINK-9373][state] Fix potential data losing for RocksDB...

2018-05-16 Thread sihuazhou
Github user sihuazhou commented on the issue: https://github.com/apache/flink/pull/6020 FYI, I found this issue related to problem: https://github.com/facebook/rocksdb/issues/3558 ---

[jira] [Created] (FLINK-9385) Operators with two inputs should show "Records Received" in Web UI separately, rather than added together

2018-05-16 Thread Josh Lemer (JIRA)
Josh Lemer created FLINK-9385: - Summary: Operators with two inputs should show "Records Received" in Web UI separately, rather than added together Key: FLINK-9385 URL: https://issues.apache.org/jira/browse/FLINK-9385

[GitHub] flink pull request #5823: [FLINK-9008] [e2e] Implements quickstarts end to e...

2018-05-16 Thread zhangminglei
Github user zhangminglei commented on a diff in the pull request: https://github.com/apache/flink/pull/5823#discussion_r188645244 --- Diff: flink-end-to-end-tests/test-scripts/test_quickstarts.sh --- @@ -0,0 +1,123 @@ +#!/usr/bin/env bash

[jira] [Commented] (FLINK-9008) End-to-end test: Quickstarts

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477502#comment-16477502 ] ASF GitHub Bot commented on FLINK-9008: --- Github user zhangminglei commented on a diff in the pull

[jira] [Commented] (FLINK-8573) Print JobID for failed jobs

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477581#comment-16477581 ] ASF GitHub Bot commented on FLINK-8573: --- Github user azagrebin commented on the issue:

[jira] [Commented] (FLINK-9301) NotSoMiniClusterIterations job fails on travis

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477452#comment-16477452 ] ASF GitHub Bot commented on FLINK-9301: --- GitHub user azagrebin opened a pull request:

[jira] [Commented] (FLINK-9301) NotSoMiniClusterIterations job fails on travis

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477453#comment-16477453 ] ASF GitHub Bot commented on FLINK-9301: --- Github user azagrebin commented on the issue:

[GitHub] flink issue #6024: [FLINK-9301] [e2e test] Add back "not co mini cluster" te...

2018-05-16 Thread azagrebin
Github user azagrebin commented on the issue: https://github.com/apache/flink/pull/6024 cc @zentol ---

[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477592#comment-16477592 ] ASF GitHub Bot commented on FLINK-9373: --- Github user sihuazhou commented on the issue:

[jira] [Updated] (FLINK-9385) Operators with two inputs should show "Records Received" in Web UI separately, rather than added together

2018-05-16 Thread Josh Lemer (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Lemer updated FLINK-9385: -- Component/s: Webfrontend Web Client Metrics > Operators with two

[jira] [Commented] (FLINK-9008) End-to-end test: Quickstarts

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477479#comment-16477479 ] ASF GitHub Bot commented on FLINK-9008: --- Github user zhangminglei commented on a diff in the pull

[GitHub] flink pull request #5823: [FLINK-9008] [e2e] Implements quickstarts end to e...

2018-05-16 Thread zhangminglei
Github user zhangminglei commented on a diff in the pull request: https://github.com/apache/flink/pull/5823#discussion_r188638639 --- Diff: flink-end-to-end-tests/test-scripts/test_quickstarts.sh --- @@ -0,0 +1,123 @@ +#!/usr/bin/env bash

[GitHub] flink pull request #6025: Release 1.4

2018-05-16 Thread zhgao
GitHub user zhgao opened a pull request: https://github.com/apache/flink/pull/6025 Release 1.4 *Thank you very much for contributing to Apache Flink - we are happy that you want to help us improve Flink. To help the community review your contribution in the best possible way,

[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477573#comment-16477573 ] ASF GitHub Bot commented on FLINK-9373: --- Github user sihuazhou commented on the issue:

[jira] [Updated] (FLINK-9370) Activate distributed cache end-to-end test

2018-05-16 Thread Chesnay Schepler (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler updated FLINK-9370: Affects Version/s: (was: 1.5.0) 1.6.0 > Activate distributed

[GitHub] flink issue #6019: [FLINK-9182]async checkpoints for timer service

2018-05-16 Thread sihuazhou
Github user sihuazhou commented on the issue: https://github.com/apache/flink/pull/6019 @StefanRRichter I definitely agree with your point! The timers should be considered as keyed state is a beautiful way to go! ---

[jira] [Commented] (FLINK-9182) async checkpoints for timer service

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477360#comment-16477360 ] ASF GitHub Bot commented on FLINK-9182: --- Github user sihuazhou commented on the issue:

[jira] [Commented] (FLINK-9043) Introduce a friendly way to resume the job from externalized checkpoints automatically

2018-05-16 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477362#comment-16477362 ] Stephan Ewen commented on FLINK-9043: - Thank you for the discussion. To me, there are two essential

[GitHub] flink issue #5896: [FLINK-8286][Security] Fix kerberos security configuratio...

2018-05-16 Thread aljoscha
Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/5896 @suez1224 Did you rebase on master? I think that krb5-specific could should not be there anymore. ---

[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477393#comment-16477393 ] ASF GitHub Bot commented on FLINK-9373: --- Github user sihuazhou commented on the issue:

[GitHub] flink issue #6020: [FLINK-9373][state] Fix potential data losing for RocksDB...

2018-05-16 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/6020 Yes, but eventually it will also return `false`, which is essentially the same as waiting until the loop terminates. Anyways, I think after the loop is the nicer way. ---

[GitHub] flink issue #6020: [FLINK-9373][state] Fix potential data losing for RocksDB...

2018-05-16 Thread sihuazhou
Github user sihuazhou commented on the issue: https://github.com/apache/flink/pull/6020 I'm going to check the native implementation and see whether the `status()` is a super cheap option... ---

[jira] [Commented] (FLINK-9381) BlobServer data for a job is not getting cleaned up at JM

2018-05-16 Thread Chesnay Schepler (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477313#comment-16477313 ] Chesnay Schepler commented on FLINK-9381: - Searching for {{BlobServer#cleanupJob}} only yields

[jira] [Assigned] (FLINK-9379) HA end-to-end test failing locally

2018-05-16 Thread Kostas Kloudas (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kostas Kloudas reassigned FLINK-9379: - Assignee: Kostas Kloudas > HA end-to-end test failing locally >

[GitHub] flink pull request #5955: [FLINK-8659] Add migration itcases for broadcast s...

2018-05-16 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/5955#discussion_r188614213 --- Diff: flink-tests/src/test/java/org/apache/flink/test/checkpointing/utils/StatefulJobWBroadcastStateMigrationITCase.java --- @@ -0,0 +1,418 @@ +/*

[jira] [Commented] (FLINK-8286) Fix Flink-Yarn-Kerberos integration for FLIP-6

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477394#comment-16477394 ] ASF GitHub Bot commented on FLINK-8286: --- Github user aljoscha commented on the issue:

[GitHub] flink issue #6020: [FLINK-9373][state] Fix potential data losing for RocksDB...

2018-05-16 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/6020 Also from the RocksDB docs: `In another word, if Iterator::Valid() is true, status() is guaranteed to be OK()` ---

[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477397#comment-16477397 ] ASF GitHub Bot commented on FLINK-9373: --- Github user StefanRRichter commented on the issue:

[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477309#comment-16477309 ] ASF GitHub Bot commented on FLINK-9373: --- Github user sihuazhou commented on the issue:

[GitHub] flink issue #6020: [FLINK-9373][state] Fix potential data losing for RocksDB...

2018-05-16 Thread sihuazhou
Github user sihuazhou commented on the issue: https://github.com/apache/flink/pull/6020 The reasons that the travis given red light is unrelated... ---

[jira] [Created] (FLINK-9381) BlobServer data for a job is not getting cleaned up at JM

2018-05-16 Thread Amit Jain (JIRA)
Amit Jain created FLINK-9381: Summary: BlobServer data for a job is not getting cleaned up at JM Key: FLINK-9381 URL: https://issues.apache.org/jira/browse/FLINK-9381 Project: Flink Issue Type:

[jira] [Commented] (FLINK-9174) The type of state created in ProccessWindowFunction.proccess() is inconsistency

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477369#comment-16477369 ] ASF GitHub Bot commented on FLINK-9174: --- Github user aljoscha commented on the issue:

[GitHub] flink issue #5847: [FLINK-9174][datastream]Fix the type of state created in ...

2018-05-16 Thread aljoscha
Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/5847 Sorry for the delay @sihuazhou I have a few minor touchups that I'll apply while merging. 👍 And thanks for discovering this and fixing it. 😄 ---

[jira] [Commented] (FLINK-8918) Introduce Runtime Filter Join

2018-05-16 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477373#comment-16477373 ] Sihua Zhou commented on FLINK-8918: --- Hi [~fhueske] thank you very much for your reply, you are right

[jira] [Comment Edited] (FLINK-8918) Introduce Runtime Filter Join

2018-05-16 Thread Sihua Zhou (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477373#comment-16477373 ] Sihua Zhou edited comment on FLINK-8918 at 5/16/18 12:49 PM: - Hi [~fhueske]

[jira] [Created] (FLINK-9382) Inconsistent file storage behavior in FileCache

2018-05-16 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-9382: --- Summary: Inconsistent file storage behavior in FileCache Key: FLINK-9382 URL: https://issues.apache.org/jira/browse/FLINK-9382 Project: Flink Issue

[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477387#comment-16477387 ] ASF GitHub Bot commented on FLINK-9373: --- Github user sihuazhou commented on the issue:

[GitHub] flink issue #6020: [FLINK-9373][state] Fix potential data losing for RocksDB...

2018-05-16 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/6020 You could also in `isRocksIteratorValid` run the check only if the return value is `false` if you like the helper method to avoid people forgetting about this check. ---

[GitHub] flink issue #6020: [FLINK-9373][state] Fix potential data losing for RocksDB...

2018-05-16 Thread sihuazhou
Github user sihuazhou commented on the issue: https://github.com/apache/flink/pull/6020 @StefanRRichter No, I didn't have any performance tests yet. I think you are right! Your proposal is the way I'm going to choose. Addressing this... ---

[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477413#comment-16477413 ] ASF GitHub Bot commented on FLINK-9373: --- Github user sihuazhou commented on the issue:

[jira] [Commented] (FLINK-8918) Introduce Runtime Filter Join

2018-05-16 Thread Fabian Hueske (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477424#comment-16477424 ] Fabian Hueske commented on FLINK-8918: -- It's good to know that you were thinking about the issues I

[jira] [Commented] (FLINK-9303) Unassign partitions from Kafka client if partitions become unavailable

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477294#comment-16477294 ] ASF GitHub Bot commented on FLINK-9303: --- Github user EAlexRojas commented on a diff in the pull

[GitHub] flink pull request #5991: [FLINK-9303] [kafka] Adding support for unassign d...

2018-05-16 Thread EAlexRojas
Github user EAlexRojas commented on a diff in the pull request: https://github.com/apache/flink/pull/5991#discussion_r188596584 --- Diff: flink-connectors/flink-connector-kafka-0.9/src/main/java/org/apache/flink/streaming/connectors/kafka/internal/KafkaConsumerThread.java --- @@

[jira] [Commented] (FLINK-8918) Introduce Runtime Filter Join

2018-05-16 Thread yuqi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477359#comment-16477359 ] yuqi commented on FLINK-8918: - # Indeed as the streaming applications runs, bloom filter will definitely goes

[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477389#comment-16477389 ] ASF GitHub Bot commented on FLINK-9373: --- Github user StefanRRichter commented on the issue:

[GitHub] flink issue #5896: [FLINK-8286][Security] Fix kerberos security configuratio...

2018-05-16 Thread aljoscha
Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/5896 @EronWright the fix is really one this: https://github.com/apache/flink/commit/ba3e2711a2091b7c0907a5caf3ea527f837a442f But I haven't yet managed to reproduce a failure on my system without the

[GitHub] flink issue #6020: [FLINK-9373][state] Fix potential data losing for RocksDB...

2018-05-16 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/6020 Oh you are right, this is confusing :-) So does this also mean the status flag is cleared when we simple continue iterating and only check in the end? ---

[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477414#comment-16477414 ] ASF GitHub Bot commented on FLINK-9373: --- Github user sihuazhou commented on the issue:

[jira] [Created] (FLINK-9380) Failing end-to-end tests should not clean up logs

2018-05-16 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-9380: Summary: Failing end-to-end tests should not clean up logs Key: FLINK-9380 URL: https://issues.apache.org/jira/browse/FLINK-9380 Project: Flink Issue Type:

[jira] [Updated] (FLINK-9381) BlobServer data for a job is not getting cleaned up at JM

2018-05-16 Thread Chesnay Schepler (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler updated FLINK-9381: Component/s: JobManager > BlobServer data for a job is not getting cleaned up at JM >

[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477383#comment-16477383 ] ASF GitHub Bot commented on FLINK-9373: --- Github user StefanRRichter commented on the issue:

[GitHub] flink issue #6020: [FLINK-9373][state] Fix potential data losing for RocksDB...

2018-05-16 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/6020 A quick general question: could you observe any performance impact from calling the `status()` method in the loops. It looks like a native method and I am not sure that it is inexpensive.

[GitHub] flink issue #5735: [FLINK-9036] [core] Add default values to State Descripto...

2018-05-16 Thread aljoscha
Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/5735 I think I'd favour the second approach because the default value is closer to where the return value is used. If it's define "far away" when defining the state descriptor it's easier for code to

[jira] [Commented] (FLINK-9036) Add default value via suppliers

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477382#comment-16477382 ] ASF GitHub Bot commented on FLINK-9036: --- Github user aljoscha commented on the issue:

[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477399#comment-16477399 ] ASF GitHub Bot commented on FLINK-9373: --- Github user StefanRRichter commented on the issue:

[jira] [Commented] (FLINK-6336) Placement Constraints for Mesos

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477404#comment-16477404 ] ASF GitHub Bot commented on FLINK-6336: --- Github user rsingh2411 commented on the issue:

[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477412#comment-16477412 ] ASF GitHub Bot commented on FLINK-9373: --- Github user sihuazhou commented on the issue:

[GitHub] flink issue #6020: [FLINK-9373][state] Fix potential data losing for RocksDB...

2018-05-16 Thread sihuazhou
Github user sihuazhou commented on the issue: https://github.com/apache/flink/pull/6020 I think I am a bit torn here now... ---

[jira] [Commented] (FLINK-9383) Extend DistributedCache E2E test to cover directories

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477417#comment-16477417 ] ASF GitHub Bot commented on FLINK-9383: --- GitHub user zentol opened a pull request:

[GitHub] flink pull request #6023: [FLINK-9383][runtime] Test directories in Distribu...

2018-05-16 Thread zentol
GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/6023 [FLINK-9383][runtime] Test directories in DistributedCache E2E test ## What is the purpose of the change With this PR the distributed cache end-to-end test * also covers the distributed

[jira] [Created] (FLINK-9379) HA end-to-end test failing locally

2018-05-16 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-9379: Summary: HA end-to-end test failing locally Key: FLINK-9379 URL: https://issues.apache.org/jira/browse/FLINK-9379 Project: Flink Issue Type: Bug

[jira] [Commented] (FLINK-9182) async checkpoints for timer service

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477353#comment-16477353 ] ASF GitHub Bot commented on FLINK-9182: --- Github user StefanRRichter commented on the issue:

[GitHub] flink issue #6019: [FLINK-9182]async checkpoints for timer service

2018-05-16 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/6019 @sihuazhou my plan was to integrate the timer service more closely with the keyed state backends, starting from the point that we are merging the PR for timers in RocksDB. I think timers

[jira] [Commented] (FLINK-9043) Introduce a friendly way to resume the job from externalized checkpoints automatically

2018-05-16 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477363#comment-16477363 ] Stephan Ewen commented on FLINK-9043: - [~ariskkol] I remember you asked about this feature once - how

[jira] [Commented] (FLINK-9182) async checkpoints for timer service

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477365#comment-16477365 ] ASF GitHub Bot commented on FLINK-9182: --- Github user StefanRRichter commented on the issue:

[GitHub] flink issue #6019: [FLINK-9182]async checkpoints for timer service

2018-05-16 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/6019 Yes, the only reason I did not start with this is that I first wanted to wait for the completion and merge of the RocksDB timer service to have a complete picture. ---

[jira] [Closed] (FLINK-9174) The type of state created in ProccessWindowFunction.proccess() is inconsistency

2018-05-16 Thread Aljoscha Krettek (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aljoscha Krettek closed FLINK-9174. --- Resolution: Fixed Fix Version/s: (was: 1.5.0) 1.5.1

[GitHub] flink issue #6020: [FLINK-9373][state] Fix potential data losing for RocksDB...

2018-05-16 Thread sihuazhou
Github user sihuazhou commented on the issue: https://github.com/apache/flink/pull/6020 @StefanRRichter NO, I think that couldn't fix this issue, the problem here is that even `iterator.isValid()` return `true`, there may also some internal error in RocksDB. What do you think? ---

[jira] [Commented] (FLINK-8286) Fix Flink-Yarn-Kerberos integration for FLIP-6

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477391#comment-16477391 ] ASF GitHub Bot commented on FLINK-8286: --- Github user aljoscha commented on the issue:

[jira] [Commented] (FLINK-9373) Fix potential data losing for RocksDBBackend

2018-05-16 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477406#comment-16477406 ] ASF GitHub Bot commented on FLINK-9373: --- Github user sihuazhou commented on the issue:

[jira] [Updated] (FLINK-9383) Extend DistributedCache E2E test to cover directories

2018-05-16 Thread Chesnay Schepler (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler updated FLINK-9383: Issue Type: Improvement (was: Bug) > Extend DistributedCache E2E test to cover directories

[GitHub] flink issue #6020: [FLINK-9373][state] Fix potential data losing for RocksDB...

2018-05-16 Thread sihuazhou
Github user sihuazhou commented on the issue: https://github.com/apache/flink/pull/6020 I think that is the incorrect one, If I'm not confused by the wiki's content... ---

[GitHub] flink issue #3744: [FLINK-6336] Initial commit of mesos placement constraint...

2018-05-16 Thread rsingh2411
Github user rsingh2411 commented on the issue: https://github.com/apache/flink/pull/3744 Getting similar error, whileadding Unique host constraint. How did you fix that imestamp=2018-05-16 10:52:34,105 LogLevel=ERROR ThreadId=[flink-akka.actor.default-dispatcher-3]

[jira] [Created] (FLINK-9383) Extend DistributedCache E2E test to cover directories

2018-05-16 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-9383: --- Summary: Extend DistributedCache E2E test to cover directories Key: FLINK-9383 URL: https://issues.apache.org/jira/browse/FLINK-9383 Project: Flink

[jira] [Closed] (FLINK-9369) DistributedCache returns parent directory for directories

2018-05-16 Thread Chesnay Schepler (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler closed FLINK-9369. --- Resolution: Invalid > DistributedCache returns parent directory for directories >

<    1   2   3   >