[jira] [Commented] (FLINK-8476) ConfigConstants#DEFAULT_HA_JOB_MANAGER_PORT unused

2018-01-23 Thread Till Rohrmann (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335499#comment-16335499 ] Till Rohrmann commented on FLINK-8476: -- Yes indeed [~Zentol]. I think we can remove

[jira] [Commented] (FLINK-8484) Kinesis consumer re-reads closed shards on job restart

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335479#comment-16335479 ] ASF GitHub Bot commented on FLINK-8484: --- GitHub user pluppens opened a pull request:

[GitHub] flink pull request #5337: [FLINK-8484][flink-kinesis-connector] Ensure a Kin...

2018-01-23 Thread pluppens
GitHub user pluppens opened a pull request: https://github.com/apache/flink/pull/5337 [FLINK-8484][flink-kinesis-connector] Ensure a Kinesis consumer snapshot restoration is able to handle recently closed shards FLINK-8484: ensure that a state change in the StreamShardMetadata

[GitHub] flink issue #5331: [FLINK-8473][webUI] Improve error behavior of JarListHand...

2018-01-23 Thread uce
Github user uce commented on the issue: https://github.com/apache/flink/pull/5331 I just tried it, re-uploading after deleting the directory **does not work**. Good catch Stephan. :-) @zentol: I found `HttpRequestHandler` which handles the uploads. The handler assumes that

[jira] [Commented] (FLINK-8473) JarListHandler may fail with NPE if directory is deleted

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335485#comment-16335485 ] ASF GitHub Bot commented on FLINK-8473: --- Github user uce commented on the issue:

[jira] [Created] (FLINK-8486) Connector for SOLR

2018-01-23 Thread Davide Gesino (JIRA)
Davide Gesino created FLINK-8486: Summary: Connector for SOLR Key: FLINK-8486 URL: https://issues.apache.org/jira/browse/FLINK-8486 Project: Flink Issue Type: New Feature

[jira] [Updated] (FLINK-8486) Connector for SOLR

2018-01-23 Thread Davide Gesino (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Gesino updated FLINK-8486: - Priority: Minor (was: Major) Description: The idea is to develop a connector for apache

[jira] [Updated] (FLINK-8486) Connector for SOLR

2018-01-23 Thread Davide Gesino (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Gesino updated FLINK-8486: - Description: The idea is to develop a connector for apache Solr with similar features to those

[jira] [Commented] (FLINK-8484) Kinesis consumer re-reads closed shards on job restart

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335525#comment-16335525 ] ASF GitHub Bot commented on FLINK-8484: --- Github user tzulitai commented on a diff in the pull

[jira] [Commented] (FLINK-8484) Kinesis consumer re-reads closed shards on job restart

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335523#comment-16335523 ] ASF GitHub Bot commented on FLINK-8484: --- Github user tzulitai commented on a diff in the pull

[GitHub] flink pull request #5337: [FLINK-8484][flink-kinesis-connector] Ensure a Kin...

2018-01-23 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5337#discussion_r163175172 --- Diff: flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/FlinkKinesisConsumer.java --- @@ -210,16

[GitHub] flink pull request #5337: [FLINK-8484][flink-kinesis-connector] Ensure a Kin...

2018-01-23 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5337#discussion_r163176506 --- Diff: flink-connectors/flink-connector-kinesis/src/test/java/org/apache/flink/streaming/connectors/kinesis/FlinkKinesisConsumerTest.java --- @@ -515,6

[GitHub] flink pull request #5337: [FLINK-8484][flink-kinesis-connector] Ensure a Kin...

2018-01-23 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5337#discussion_r163175349 --- Diff: flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/FlinkKinesisConsumer.java --- @@ -267,6

[jira] [Commented] (FLINK-8484) Kinesis consumer re-reads closed shards on job restart

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335524#comment-16335524 ] ASF GitHub Bot commented on FLINK-8484: --- Github user tzulitai commented on a diff in the pull

[jira] [Created] (FLINK-8487) State loss after multiple restart attempts

2018-01-23 Thread Fabian Hueske (JIRA)
Fabian Hueske created FLINK-8487: Summary: State loss after multiple restart attempts Key: FLINK-8487 URL: https://issues.apache.org/jira/browse/FLINK-8487 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-8488) Dispatcher does not recover jobs

2018-01-23 Thread Gary Yao (JIRA)
Gary Yao created FLINK-8488: --- Summary: Dispatcher does not recover jobs Key: FLINK-8488 URL: https://issues.apache.org/jira/browse/FLINK-8488 Project: Flink Issue Type: Bug Components:

[jira] [Updated] (FLINK-8488) Dispatcher does not recover jobs

2018-01-23 Thread Gary Yao (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Yao updated FLINK-8488: Description: Dispatcher does not recover jobs on failover (FLIP-6 mode). *Steps to reproduce*: #

[jira] [Updated] (FLINK-8488) Dispatcher does not recover jobs

2018-01-23 Thread Gary Yao (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Yao updated FLINK-8488: Description: Dispatcher does not recover jobs on failover (FLIP-6 mode). *Steps to reproduce*: #

[jira] [Updated] (FLINK-8488) Dispatcher does not recover jobs

2018-01-23 Thread Gary Yao (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Yao updated FLINK-8488: Labels: flip-6 (was: ) Description: Dispatcher does not recover jobs on failover (FLIP-6).

[GitHub] flink issue #5323: [FLINK-8441] [State Backend] [RocksDB] change RocksDBList...

2018-01-23 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/5323 Thanks for your contribution, I will merge this with some small cleanup. ---

[jira] [Commented] (FLINK-8487) State loss after multiple restart attempts

2018-01-23 Thread Stefan Richter (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335551#comment-16335551 ] Stefan Richter commented on FLINK-8487: --- Afaik [~aljoscha] already fixed this in FLINK-7783? >

[jira] [Created] (FLINK-8489) Data is not emitted by second ElasticSearch connector

2018-01-23 Thread Fabian Hueske (JIRA)
Fabian Hueske created FLINK-8489: Summary: Data is not emitted by second ElasticSearch connector Key: FLINK-8489 URL: https://issues.apache.org/jira/browse/FLINK-8489 Project: Flink Issue

[GitHub] flink pull request #5323: [FLINK-8441] [State Backend] [RocksDB] change Rock...

2018-01-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/5323 ---

[jira] [Commented] (FLINK-8441) serialize values and value separator directly to stream in RocksDBListState

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335598#comment-16335598 ] ASF GitHub Bot commented on FLINK-8441: --- Github user asfgit closed the pull request at:

[jira] [Updated] (FLINK-8484) Kinesis consumer re-reads closed shards on job restart

2018-01-23 Thread Tzu-Li (Gordon) Tai (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tzu-Li (Gordon) Tai updated FLINK-8484: --- Priority: Blocker (was: Major) > Kinesis consumer re-reads closed shards on job

[jira] [Commented] (FLINK-8484) Kinesis consumer re-reads closed shards on job restart

2018-01-23 Thread Tzu-Li (Gordon) Tai (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335528#comment-16335528 ] Tzu-Li (Gordon) Tai commented on FLINK-8484: I think we should make this a blocker for 1.4.1 &

[jira] [Assigned] (FLINK-8484) Kinesis consumer re-reads closed shards on job restart

2018-01-23 Thread Tzu-Li (Gordon) Tai (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tzu-Li (Gordon) Tai reassigned FLINK-8484: -- Assignee: Philip Luppens > Kinesis consumer re-reads closed shards on job

[jira] [Assigned] (FLINK-8484) Kinesis consumer re-reads closed shards on job restart

2018-01-23 Thread Tzu-Li (Gordon) Tai (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tzu-Li (Gordon) Tai reassigned FLINK-8484: -- Assignee: Philip Luppens (was: Tzu-Li (Gordon) Tai) > Kinesis consumer

[jira] [Updated] (FLINK-8488) Dispatcher does not recover jobs

2018-01-23 Thread Gary Yao (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Yao updated FLINK-8488: Description: Dispatcher does not recover jobs on failover (FLIP-6 mode). *Steps to reproduce*: #

[jira] [Updated] (FLINK-8488) Dispatcher does not recover jobs

2018-01-23 Thread Gary Yao (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Yao updated FLINK-8488: Description: Dispatcher does not recover jobs on failover (FLIP-6 mode). *Steps to reproduce*: #

[jira] [Updated] (FLINK-8488) Dispatcher does not recover Jobs

2018-01-23 Thread Gary Yao (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Yao updated FLINK-8488: Summary: Dispatcher does not recover Jobs (was: Dispatcher does not recover jobs) > Dispatcher does not

[jira] [Commented] (FLINK-8441) serialize values and value separator directly to stream in RocksDBListState

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335595#comment-16335595 ] ASF GitHub Bot commented on FLINK-8441: --- Github user StefanRRichter commented on the issue:

[jira] [Updated] (FLINK-8484) Kinesis consumer re-reads closed shards on job restart

2018-01-23 Thread Tzu-Li (Gordon) Tai (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tzu-Li (Gordon) Tai updated FLINK-8484: --- Fix Version/s: 1.4.1 1.5.0 1.3.3 > Kinesis

[jira] [Updated] (FLINK-8484) Kinesis consumer re-reads closed shards on job restart

2018-01-23 Thread Tzu-Li (Gordon) Tai (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tzu-Li (Gordon) Tai updated FLINK-8484: --- Affects Version/s: 1.4.0 > Kinesis consumer re-reads closed shards on job restart >

[jira] [Assigned] (FLINK-8484) Kinesis consumer re-reads closed shards on job restart

2018-01-23 Thread Tzu-Li (Gordon) Tai (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tzu-Li (Gordon) Tai reassigned FLINK-8484: -- Assignee: Tzu-Li (Gordon) Tai (was: Philip Luppens) > Kinesis consumer

[jira] [Commented] (FLINK-8101) Elasticsearch 6.x support

2018-01-23 Thread Christophe Jolif (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335694#comment-16335694 ] Christophe Jolif commented on FLINK-8101: - It seems Elasticsearch is slowly moving away from the

[GitHub] flink pull request #5312: [FLINK-8344][flip6] Add support for HA to RestClus...

2018-01-23 Thread GJL
Github user GJL commented on a diff in the pull request: https://github.com/apache/flink/pull/5312#discussion_r163227450 --- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java --- @@ -596,11 +596,11 @@ public int run(String[] args) throws

[jira] [Commented] (FLINK-8344) Add support for HA to RestClusterClient

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335699#comment-16335699 ] ASF GitHub Bot commented on FLINK-8344: --- Github user GJL commented on a diff in the pull request:

[jira] [Comment Edited] (FLINK-8101) Elasticsearch 6.x support

2018-01-23 Thread Christophe Jolif (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335694#comment-16335694 ] Christophe Jolif edited comment on FLINK-8101 at 1/23/18 12:25 PM: --- It

[jira] [Commented] (FLINK-8484) Kinesis consumer re-reads closed shards on job restart

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335703#comment-16335703 ] ASF GitHub Bot commented on FLINK-8484: --- Github user tzulitai commented on the issue:

[GitHub] flink issue #5337: [FLINK-8484][flink-kinesis-connector] Ensure a Kinesis co...

2018-01-23 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5337 @pluppens yes, I think that would be the proper solution here. ---

[jira] [Commented] (FLINK-7386) Flink Elasticsearch 5 connector is not compatible with Elasticsearch 5.2+ client

2018-01-23 Thread Christophe Jolif (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335704#comment-16335704 ] Christophe Jolif commented on FLINK-7386: - Similarly to my comment on FLINK-8101 wouldn't it make

[jira] [Commented] (FLINK-8484) Kinesis consumer re-reads closed shards on job restart

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335726#comment-16335726 ] ASF GitHub Bot commented on FLINK-8484: --- Github user pluppens commented on the issue:

[GitHub] flink issue #5337: [FLINK-8484][flink-kinesis-connector] Ensure a Kinesis co...

2018-01-23 Thread pluppens
Github user pluppens commented on the issue: https://github.com/apache/flink/pull/5337 Ok, that makes sense to me. Give me a bit to cook up both the new test and the new approach, and I'll update the PR. Thank you very much for the comments! ---

[jira] [Created] (FLINK-8492) Fix unsupported exception for udtf with multi calc

2018-01-23 Thread Hequn Cheng (JIRA)
Hequn Cheng created FLINK-8492: -- Summary: Fix unsupported exception for udtf with multi calc Key: FLINK-8492 URL: https://issues.apache.org/jira/browse/FLINK-8492 Project: Flink Issue Type: Bug

[jira] [Updated] (FLINK-8492) Fix unsupported exception for udtf with multi calc

2018-01-23 Thread Hequn Cheng (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hequn Cheng updated FLINK-8492: --- Description: Considering the following test, unsupported exception will be thrown due to  multi calc

[jira] [Updated] (FLINK-8492) Fix unsupported exception for udtf with multi calc

2018-01-23 Thread Hequn Cheng (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hequn Cheng updated FLINK-8492: --- Description: Considering the following test, unsupported exception will be thrown due to  multi calc

[jira] [Created] (FLINK-8493) Integrate queryable state with Flip-6

2018-01-23 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-8493: Summary: Integrate queryable state with Flip-6 Key: FLINK-8493 URL: https://issues.apache.org/jira/browse/FLINK-8493 Project: Flink Issue Type: Improvement

[GitHub] flink issue #5331: [FLINK-8473][webUI] Improve error behavior of JarListHand...

2018-01-23 Thread zentol
Github user zentol commented on the issue: https://github.com/apache/flink/pull/5331 I've updated the PR. All handlers that access the jar storage directory now check whether it exists and regenerate it if necessary with appropriate logging messages. The logging and directory

[jira] [Assigned] (FLINK-8476) ConfigConstants#DEFAULT_HA_JOB_MANAGER_PORT unused

2018-01-23 Thread Chesnay Schepler (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler reassigned FLINK-8476: --- Assignee: Chesnay Schepler > ConfigConstants#DEFAULT_HA_JOB_MANAGER_PORT unused >

[jira] [Updated] (FLINK-8476) ConfigConstants#DEFAULT_HA_JOB_MANAGER_PORT unused

2018-01-23 Thread Chesnay Schepler (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler updated FLINK-8476: Fix Version/s: 1.5.0 > ConfigConstants#DEFAULT_HA_JOB_MANAGER_PORT unused >

[jira] [Created] (FLINK-8490) Allow custom docker parameters for docker tasks on Mesos

2018-01-23 Thread JIRA
Jörg Schad created FLINK-8490: - Summary: Allow custom docker parameters for docker tasks on Mesos Key: FLINK-8490 URL: https://issues.apache.org/jira/browse/FLINK-8490 Project: Flink Issue Type:

[jira] [Commented] (FLINK-8473) JarListHandler may fail with NPE if directory is deleted

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335660#comment-16335660 ] ASF GitHub Bot commented on FLINK-8473: --- Github user zentol commented on the issue:

[jira] [Created] (FLINK-8491) Deprecate ConfigConstants#HA_MODE

2018-01-23 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-8491: --- Summary: Deprecate ConfigConstants#HA_MODE Key: FLINK-8491 URL: https://issues.apache.org/jira/browse/FLINK-8491 Project: Flink Issue Type:

[jira] [Commented] (FLINK-8490) Allow custom docker parameters for docker tasks on Mesos

2018-01-23 Thread JIRA
[ https://issues.apache.org/jira/browse/FLINK-8490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335667#comment-16335667 ] Jörg Schad commented on FLINK-8490: --- I would propose the following parameter name:

[jira] [Closed] (FLINK-8491) Deprecate ConfigConstants#HA_MODE

2018-01-23 Thread Chesnay Schepler (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler closed FLINK-8491. --- Resolution: Duplicate Assignee: (was: Chesnay Schepler) Fix Version/s:

[GitHub] flink pull request #5338: [FLINK-8476][config][HA] Deprecate HA config const...

2018-01-23 Thread zentol
GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/5338 [FLINK-8476][config][HA] Deprecate HA config constants ## What is the purpose of the change This PR deprecates multiple HA-related `ConfigConstants`. ## Brief change log

[jira] [Commented] (FLINK-8476) ConfigConstants#DEFAULT_HA_JOB_MANAGER_PORT unused

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335676#comment-16335676 ] ASF GitHub Bot commented on FLINK-8476: --- GitHub user zentol opened a pull request:

[jira] [Closed] (FLINK-8158) Rowtime window inner join emits late data

2018-01-23 Thread Hequn Cheng (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hequn Cheng closed FLINK-8158. -- Resolution: Won't Do > Rowtime window inner join emits late data >

[GitHub] flink pull request #5337: [FLINK-8484][flink-kinesis-connector] Ensure a Kin...

2018-01-23 Thread pluppens
Github user pluppens commented on a diff in the pull request: https://github.com/apache/flink/pull/5337#discussion_r163226460 --- Diff: flink-connectors/flink-connector-kinesis/src/test/java/org/apache/flink/streaming/connectors/kinesis/FlinkKinesisConsumerTest.java --- @@ -515,6

[jira] [Commented] (FLINK-8484) Kinesis consumer re-reads closed shards on job restart

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335689#comment-16335689 ] ASF GitHub Bot commented on FLINK-8484: --- Github user pluppens commented on a diff in the pull

[GitHub] flink issue #5337: [FLINK-8484][flink-kinesis-connector] Ensure a Kinesis co...

2018-01-23 Thread pluppens
Github user pluppens commented on the issue: https://github.com/apache/flink/pull/5337 Ok, so you'd prefer to synchronize the state of the retrieve shard against the stored shards by comparing its stream name and shard id, before doing the containsKey() check? ---

[jira] [Commented] (FLINK-8484) Kinesis consumer re-reads closed shards on job restart

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335691#comment-16335691 ] ASF GitHub Bot commented on FLINK-8484: --- Github user pluppens commented on the issue:

[GitHub] flink pull request #5326: [FLINK-8365] [State Backend] Relax List type in He...

2018-01-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/5326 ---

[GitHub] flink pull request #5325: [FLINK-8469] [State Backend] [RocksDB] relocate an...

2018-01-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/5325 ---

[GitHub] flink pull request #5300: [FLINK-8411] [State Backends] HeapListState#add(nu...

2018-01-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/5300 ---

[jira] [Commented] (FLINK-8411) HeapListState#add(null) will wipe out entire list state

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335753#comment-16335753 ] ASF GitHub Bot commented on FLINK-8411: --- Github user asfgit closed the pull request at:

[GitHub] flink pull request #5339: [FLINK-8493] [flip6] Integrate queryable state wit...

2018-01-23 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/5339 [FLINK-8493] [flip6] Integrate queryable state with Flip-6 ## What is the purpose of the change Adapt KvStateRegistry to accept multiple KvStateRegistryListeners. Introduce the

[jira] [Commented] (FLINK-8365) Relax List type in HeapListState and HeapKeyedStateBackend

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335752#comment-16335752 ] ASF GitHub Bot commented on FLINK-8365: --- Github user asfgit closed the pull request at:

[jira] [Commented] (FLINK-8493) Integrate queryable state with Flip-6

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335755#comment-16335755 ] ASF GitHub Bot commented on FLINK-8493: --- GitHub user tillrohrmann opened a pull request:

[jira] [Commented] (FLINK-8469) relocate and unify RocksDB option params in RocksDBPerformanceTest

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335754#comment-16335754 ] ASF GitHub Bot commented on FLINK-8469: --- Github user asfgit closed the pull request at:

[jira] [Updated] (FLINK-8492) Fix unsupported exception for udtf with multi calc

2018-01-23 Thread Hequn Cheng (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hequn Cheng updated FLINK-8492: --- Description: Considering the following test, unsupported exception will be thrown due to  multi calc

[GitHub] flink issue #5337: [FLINK-8484][flink-kinesis-connector] Ensure a Kinesis co...

2018-01-23 Thread pluppens
Github user pluppens commented on the issue: https://github.com/apache/flink/pull/5337 Just a small remark - from what I understood, the only property that *can* change is the endingSequenceNumber - all other state should be considered as 'set once', so there should be no point in

[jira] [Commented] (FLINK-8484) Kinesis consumer re-reads closed shards on job restart

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335762#comment-16335762 ] ASF GitHub Bot commented on FLINK-8484: --- Github user pluppens commented on the issue:

[GitHub] flink issue #5337: [FLINK-8484][flink-kinesis-connector] Ensure a Kinesis co...

2018-01-23 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5337 @pluppens yes, that sounds correct. `parentShardId` `adjacentParentShardId` `startingHashKey` `endingHashKey` `startingSequenceNumber` these should all be fixed once the

[jira] [Commented] (FLINK-8484) Kinesis consumer re-reads closed shards on job restart

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335765#comment-16335765 ] ASF GitHub Bot commented on FLINK-8484: --- Github user tzulitai commented on the issue:

[jira] [Closed] (FLINK-8469) relocate and unify RocksDB option params in RocksDBPerformanceTest

2018-01-23 Thread Stefan Richter (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter closed FLINK-8469. - Resolution: Fixed Merged in 1c9c1e36c1dd7b3f2a160216e405302d7854c148 . > relocate and unify

[jira] [Closed] (FLINK-8411) HeapListState#add(null) will wipe out entire list state

2018-01-23 Thread Stefan Richter (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter closed FLINK-8411. - Resolution: Fixed Merged in e157cfa77f83608a0cd6d7d41a96edb0ca1f97f6 . > HeapListState#add(null)

[jira] [Closed] (FLINK-8365) Relax List type in HeapListState and HeapKeyedStateBackend

2018-01-23 Thread Stefan Richter (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter closed FLINK-8365. - Resolution: Fixed Merged in e075da5d5eb0f5ae8c394ea0c549f9dbce28fcf3 . > Relax List type in

[jira] [Closed] (FLINK-8441) serialize values and value separator directly to stream in RocksDBListState

2018-01-23 Thread Stefan Richter (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Richter closed FLINK-8441. - Resolution: Fixed Merged in ce25688bac1c1ecfa53a03ab1857bb82963b0696 . > serialize values and

[jira] [Updated] (FLINK-8492) [FLINK-8492][table] Fix calc cost bug

2018-01-23 Thread Hequn Cheng (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hequn Cheng updated FLINK-8492: --- Summary: [FLINK-8492][table] Fix calc cost bug (was: Fix unsupported exception for udtf with multi

[GitHub] flink pull request #5347: [FLINK-8492][table] Fix calc cost bug

2018-01-23 Thread hequn8128
GitHub user hequn8128 opened a pull request: https://github.com/apache/flink/pull/5347 [FLINK-8492][table] Fix calc cost bug ## What is the purpose of the change Fix calc cost bug. Currently, unsupported exception will be thrown when multi calc existing between

[jira] [Commented] (FLINK-8492) [FLINK-8492][table] Fix calc cost bug

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16336937#comment-16336937 ] ASF GitHub Bot commented on FLINK-8492: --- GitHub user hequn8128 opened a pull request:

[jira] [Commented] (FLINK-7129) Dynamically changing patterns

2018-01-23 Thread aitozi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16336981#comment-16336981 ] aitozi commented on FLINK-7129: --- Hi,can i ask the progress of this feature ? > Dynamically changing

[jira] [Created] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

2018-01-23 Thread yanxiaobin (JIRA)
yanxiaobin created FLINK-8500: - Summary: Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher) Key: FLINK-8500 URL: https://issues.apache.org/jira/browse/FLINK-8500 Project: Flink

[jira] [Commented] (FLINK-8492) Fix unsupported exception for udtf with multi calc

2018-01-23 Thread Hequn Cheng (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16336778#comment-16336778 ] Hequn Cheng commented on FLINK-8492: Cool, It's an excellent solution! I will submit a pr soon. Thanks

[jira] [Resolved] (FLINK-5227) Add warning to include flink-table in job fat jars

2018-01-23 Thread Fabian Hueske (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabian Hueske resolved FLINK-5227. -- Resolution: Fixed Fix Version/s: (was: 1.1.5) Warning has been added to the docs

[jira] [Updated] (FLINK-5322) Clean up yarn configuration documentation

2018-01-23 Thread Chesnay Schepler (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler updated FLINK-5322: Fix Version/s: (was: 1.1.5) > Clean up yarn configuration documentation >

[jira] [Commented] (FLINK-8493) Integrate queryable state with Flip-6

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16336152#comment-16336152 ] ASF GitHub Bot commented on FLINK-8493: --- Github user GJL commented on the issue:

[GitHub] flink issue #5339: [FLINK-8493] [flip6] Integrate queryable state with Flip-...

2018-01-23 Thread GJL
Github user GJL commented on the issue: https://github.com/apache/flink/pull/5339 ui ---

[jira] [Created] (FLINK-8499) Kryo must not be child-first loaded

2018-01-23 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-8499: --- Summary: Kryo must not be child-first loaded Key: FLINK-8499 URL: https://issues.apache.org/jira/browse/FLINK-8499 Project: Flink Issue Type: Bug

[jira] [Updated] (FLINK-6209) StreamPlanEnvironment always has a parallelism of 1

2018-01-23 Thread Chesnay Schepler (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler updated FLINK-6209: Fix Version/s: (was: 1.2.2) > StreamPlanEnvironment always has a parallelism of 1 >

[jira] [Updated] (FLINK-5030) Support hostname verification

2018-01-23 Thread Chesnay Schepler (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler updated FLINK-5030: Fix Version/s: (was: 1.2.2) > Support hostname verification >

[jira] [Updated] (FLINK-8466) MemoryArchivist holding on to last thrown exception prevents the user code classloader from being garbage collected

2018-01-23 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen updated FLINK-8466: Fix Version/s: 1.4.1 1.5.0 > MemoryArchivist holding on to last thrown exception

[jira] [Updated] (FLINK-8466) MemoryArchivist holding on to last thrown exception prevents the user code classloader from being garbage collected

2018-01-23 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen updated FLINK-8466: Priority: Blocker (was: Major) > MemoryArchivist holding on to last thrown exception prevents the

[jira] [Updated] (FLINK-8466) ErrorInfo needs to hold Exception as SerializedThrowable

2018-01-23 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen updated FLINK-8466: Summary: ErrorInfo needs to hold Exception as SerializedThrowable (was: MemoryArchivist holding on

[GitHub] flink issue #5172: [FLINK-8275] [Security] fix keytab local path in YarnTask...

2018-01-23 Thread toggm
Github user toggm commented on the issue: https://github.com/apache/flink/pull/5172 We try to use apache flink in Kerberos secured environment and had to backport to flink 1.3.2 because of that issue. Would be good to have that PR to be able to migrate to the latest flink version.

[jira] [Commented] (FLINK-8275) Flink YARN deployment with Kerberos enabled not working

2018-01-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16337049#comment-16337049 ] ASF GitHub Bot commented on FLINK-8275: --- Github user toggm commented on the issue:

[jira] [Closed] (FLINK-4766) Migrate TaskManager configuration options

2018-01-23 Thread Chesnay Schepler (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler closed FLINK-4766. --- Resolution: Fixed Was resolved in the meantime. > Migrate TaskManager configuration options

[GitHub] flink pull request #5340: [FLINK-8475][config][docs] Integrate more ConfigOp...

2018-01-23 Thread zentol
GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/5340 [FLINK-8475][config][docs] Integrate more ConfigOptions into documentation ## What is the purpose of the change This PR significantly extends the integration of ConfigOptions into the

  1   2   >