[jira] [Commented] (FLINK-13848) Support “scheduleAtFixedRate/scheduleAtFixedDelay” in RpcEndpoint#MainThreadExecutor

2019-08-27 Thread Xiaogang Shi (Jira)
[ https://issues.apache.org/jira/browse/FLINK-13848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916617#comment-16916617 ] Xiaogang Shi commented on FLINK-13848: -- [~SleePy] Sorry that I misunderstand your second question.

[jira] [Commented] (FLINK-13848) Support “scheduleAtFixedRate/scheduleAtFixedDelay” in RpcEndpoint#MainThreadExecutor

2019-08-27 Thread Xiaogang Shi (Jira)
[ https://issues.apache.org/jira/browse/FLINK-13848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916607#comment-16916607 ] Xiaogang Shi commented on FLINK-13848: -- 1. Yes. If the task message is already sent to the actor's

[jira] [Commented] (FLINK-13848) Support “scheduleAtFixedRate/scheduleAtFixedDelay” in RpcEndpoint#MainThreadExecutor

2019-08-26 Thread Xiaogang Shi (Jira)
[ https://issues.apache.org/jira/browse/FLINK-13848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915689#comment-16915689 ] Xiaogang Shi commented on FLINK-13848: -- Hi [~SleePy], what's your detailed plan on implementing

[jira] [Commented] (FLINK-13732) Enhance JobManagerMetricGroup with FLIP-6 architecture

2019-08-15 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-13732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907856#comment-16907856 ] Xiaogang Shi commented on FLINK-13732: -- [~SleePy] Thanks for bringing up this issue. We are also

[jira] [Assigned] (FLINK-12887) Schedule UnfencedMessage would lost envelope info

2019-07-22 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi reassigned FLINK-12887: Assignee: TisonKun (was: Xiaogang Shi) > Schedule UnfencedMessage would lost envelope

[jira] [Assigned] (FLINK-12887) Schedule UnfencedMessage would lost envelope info

2019-07-22 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi reassigned FLINK-12887: Assignee: Xiaogang Shi > Schedule UnfencedMessage would lost envelope info >

[jira] [Commented] (FLINK-12887) Schedule UnfencedMessage would lost envelope info

2019-07-11 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883233#comment-16883233 ] Xiaogang Shi commented on FLINK-12887: -- As described in my first comment, we are using

[jira] [Commented] (FLINK-12887) Schedule UnfencedMessage would lost envelope info

2019-06-20 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869118#comment-16869118 ] Xiaogang Shi commented on FLINK-12887: -- [~till.rohrmann] I am also very curious on the method we

[jira] [Commented] (FLINK-12887) Schedule UnfencedMessage would lost envelope info

2019-06-20 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869086#comment-16869086 ] Xiaogang Shi commented on FLINK-12887: -- Hi [~till.rohrmann], now we are using many unfenced

[jira] [Commented] (FLINK-12865) State inconsistency between RM and TM on the slot status

2019-06-17 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866148#comment-16866148 ] Xiaogang Shi commented on FLINK-12865: -- [~till.rohrmann]You are right that there is no problem with

[jira] [Commented] (FLINK-12863) Race condition between slot offerings and AllocatedSlotReport

2019-06-16 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865269#comment-16865269 ] Xiaogang Shi commented on FLINK-12863: -- Btw, i want to note that the race condition may not

[jira] [Commented] (FLINK-12863) Race condition between slot offerings and AllocatedSlotReport

2019-06-16 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865239#comment-16865239 ] Xiaogang Shi commented on FLINK-12863: -- I think a similar problem happens in the heartbeats between

[jira] [Commented] (FLINK-12761) Fine grained resource management

2019-06-06 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857448#comment-16857448 ] Xiaogang Shi commented on FLINK-12761: -- I think it's a good idea to enable fine-grained resource

[jira] [Created] (FLINK-12285) Memory leak in SavepointITCase and SavepointMigrationTestBase

2019-04-22 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-12285: Summary: Memory leak in SavepointITCase and SavepointMigrationTestBase Key: FLINK-12285 URL: https://issues.apache.org/jira/browse/FLINK-12285 Project: Flink

[jira] [Created] (FLINK-12284) InputBufferPoolUsage is incorrect in credit-based network control flow

2019-04-22 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-12284: Summary: InputBufferPoolUsage is incorrect in credit-based network control flow Key: FLINK-12284 URL: https://issues.apache.org/jira/browse/FLINK-12284 Project:

[jira] [Commented] (FLINK-10052) Tolerate temporarily suspended ZooKeeper connections

2019-04-12 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816118#comment-16816118 ] Xiaogang Shi commented on FLINK-10052: -- What's the status of the issue? [~Wosinsan]

[jira] [Commented] (FLINK-10333) Rethink ZooKeeper based stores (SubmittedJobGraph, MesosWorker, CompletedCheckpoints)

2018-11-28 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16702604#comment-16702604 ] Xiaogang Shi commented on FLINK-10333: -- I think so. We can wrap all the access to ZK in the

[jira] [Commented] (FLINK-10333) Rethink ZooKeeper based stores (SubmittedJobGraph, MesosWorker, CompletedCheckpoints)

2018-11-28 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16701821#comment-16701821 ] Xiaogang Shi commented on FLINK-10333: -- hi [~StephanEwen], I think the key point to achieve

[jira] [Commented] (FLINK-10333) Rethink ZooKeeper based stores (SubmittedJobGraph, MesosWorker, CompletedCheckpoints)

2018-11-27 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16701495#comment-16701495 ] Xiaogang Shi commented on FLINK-10333: -- Hi [~till.rohrmann] Our cluster is suffering from unstable

[jira] [Commented] (FLINK-10714) java.lang.IndexOutOfBoundsException when creating a heap backend snapshot

2018-10-29 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668009#comment-16668009 ] Xiaogang Shi commented on FLINK-10714: -- [~cmick] I came across a similar problem before. It seems 

[jira] [Commented] (FLINK-6557) RocksDbStateBackendTest fails on Windows

2017-05-14 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009910#comment-16009910 ] Xiaogang Shi commented on FLINK-6557: - [~Zentol] Thanks a lot for pointing it out. As far as i know,

[jira] [Commented] (FLINK-6545) Make incremental checkpoints externalizable

2017-05-12 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007785#comment-16007785 ] Xiaogang Shi commented on FLINK-6545: - I prefer not to use `SavepointSerializer` to

[jira] [Commented] (FLINK-6467) Potential lack of synchronization w.r.t. newSstFiles in RocksDBKeyedStateBackend#releaseResources()

2017-05-09 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002329#comment-16002329 ] Xiaogang Shi commented on FLINK-6467: - [~te...@apache.org] Thanks for pointing out this problem. It's

[jira] [Assigned] (FLINK-6504) Lack of synchronization on materializedSstFiles in RocksDBKEyedStateBackend

2017-05-09 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi reassigned FLINK-6504: --- Assignee: Xiaogang Shi > Lack of synchronization on materializedSstFiles in

[jira] [Created] (FLINK-6364) Implement incremental checkpointing in RocksDBStateBackend

2017-04-23 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-6364: --- Summary: Implement incremental checkpointing in RocksDBStateBackend Key: FLINK-6364 URL: https://issues.apache.org/jira/browse/FLINK-6364 Project: Flink Issue

[jira] [Closed] (FLINK-6014) Allow the registration of state objects in checkpoints

2017-04-23 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi closed FLINK-6014. --- Resolution: Fixed Fixed via 218bed8b8e49b0e4c61c61f696a8f010eafea1b7 and

[jira] [Created] (FLINK-6284) Incorrect sorting of completed checkpoints in ZooKeeperCompletedCheckpointStore

2017-04-09 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-6284: --- Summary: Incorrect sorting of completed checkpoints in ZooKeeperCompletedCheckpointStore Key: FLINK-6284 URL: https://issues.apache.org/jira/browse/FLINK-6284 Project:

[jira] [Commented] (FLINK-3089) State API Should Support Data Expiration

2017-04-09 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962347#comment-15962347 ] Xiaogang Shi commented on FLINK-3089: - [~aljoscha] In current implementation, each RocksDB timer is

[jira] [Commented] (FLINK-6219) Add a state backend which supports sorting

2017-03-31 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950388#comment-15950388 ] Xiaogang Shi commented on FLINK-6219: - I prefer to use sorted states (e.g., {{SortedMapState}}) rather

[jira] [Commented] (FLINK-6178) Allow upgrades to state serializers

2017-03-28 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944776#comment-15944776 ] Xiaogang Shi commented on FLINK-6178: - [~tzulitai] Thanks a lot for your quick response. The changes

[jira] [Commented] (FLINK-6178) Allow upgrades to state serializers

2017-03-28 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944743#comment-15944743 ] Xiaogang Shi commented on FLINK-6178: - The idea that allowing the upgrades to state serializers is

[jira] [Assigned] (FLINK-6096) Refactor the migration of old versioned savepoints

2017-03-17 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi reassigned FLINK-6096: --- Assignee: Xiaogang Shi > Refactor the migration of old versioned savepoints >

[jira] [Updated] (FLINK-6096) Refactor the migration of old versioned savepoints

2017-03-17 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi updated FLINK-6096: Component/s: State Backends, Checkpointing > Refactor the migration of old versioned savepoints >

[jira] [Created] (FLINK-6096) Refactor the migration of old versioned savepoints

2017-03-17 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-6096: --- Summary: Refactor the migration of old versioned savepoints Key: FLINK-6096 URL: https://issues.apache.org/jira/browse/FLINK-6096 Project: Flink Issue Type:

[jira] [Commented] (FLINK-6055) Supported setting timers on a Non-Keyed Stream

2017-03-14 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15925575#comment-15925575 ] Xiaogang Shi commented on FLINK-6055: - I think it's very challenging because the storing and restoring

[jira] [Updated] (FLINK-6034) Add KeyedStateHandle for the snapshots in keyed streams

2017-03-13 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi updated FLINK-6034: Description: Currently, the only type of the snapshots in keyed streams is

[jira] [Created] (FLINK-6034) Add KeyedStateHandle for the snapshots in keyed streams

2017-03-13 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-6034: --- Summary: Add KeyedStateHandle for the snapshots in keyed streams Key: FLINK-6034 URL: https://issues.apache.org/jira/browse/FLINK-6034 Project: Flink Issue

[jira] [Created] (FLINK-6027) Ignore the exception thrown by the subsuming of old completed checkpoints

2017-03-13 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-6027: --- Summary: Ignore the exception thrown by the subsuming of old completed checkpoints Key: FLINK-6027 URL: https://issues.apache.org/jira/browse/FLINK-6027 Project: Flink

[jira] [Created] (FLINK-6014) Allow the registration of state objects in checkpoints

2017-03-09 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-6014: --- Summary: Allow the registration of state objects in checkpoints Key: FLINK-6014 URL: https://issues.apache.org/jira/browse/FLINK-6014 Project: Flink Issue

[jira] [Assigned] (FLINK-5053) Incremental / lightweight snapshots for checkpoints

2017-03-08 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi reassigned FLINK-5053: --- Assignee: Xiaogang Shi > Incremental / lightweight snapshots for checkpoints >

[jira] [Assigned] (FLINK-5917) Remove MapState.size()

2017-03-02 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi reassigned FLINK-5917: --- Assignee: Xiaogang Shi > Remove MapState.size() > -- > >

[jira] [Commented] (FLINK-5917) Remove MapState.size()

2017-03-02 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891970#comment-15891970 ] Xiaogang Shi commented on FLINK-5917: - +1 to remove the {{size()}} method due to the cost

[jira] [Commented] (FLINK-5925) Clean up extracted RocksDB JNI library

2017-02-28 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889462#comment-15889462 ] Xiaogang Shi commented on FLINK-5925: - +1 for the cleanup mechanism. Each component (e.g.

[jira] [Commented] (FLINK-5917) Remove MapState.size()

2017-02-27 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15887094#comment-15887094 ] Xiaogang Shi commented on FLINK-5917: - [~StephanEwen] We just use the cache to avoid the costly

[jira] [Commented] (FLINK-5917) Remove MapState.size()

2017-02-26 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885019#comment-15885019 ] Xiaogang Shi commented on FLINK-5917: - I think it's okay to remove this method. But a better choice is

[jira] [Closed] (FLINK-5790) Use list types when ListStateDescriptor extends StateDescriptor

2017-02-23 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi closed FLINK-5790. --- Resolution: Fixed Fixed via d47446cafffe0d34d89488f6eb860aa139ceb3f1 > Use list types when

[jira] [Closed] (FLINK-5863) Unify the serialization of queryable list states in different backends

2017-02-23 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi closed FLINK-5863. --- Resolution: Fixed Now that we are refactoring the queryable states, we can make the changes then. >

[jira] [Updated] (FLINK-5865) Throw original exception in states

2017-02-21 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi updated FLINK-5865: Summary: Throw original exception in states (was: Throw original exception in RocksDB states) >

[jira] [Created] (FLINK-5865) Throw original exception in RocksDB states

2017-02-21 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-5865: --- Summary: Throw original exception in RocksDB states Key: FLINK-5865 URL: https://issues.apache.org/jira/browse/FLINK-5865 Project: Flink Issue Type:

[jira] [Assigned] (FLINK-5863) Unify the serialization of queryable list states in different backends

2017-02-21 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi reassigned FLINK-5863: --- Assignee: Xiaogang Shi > Unify the serialization of queryable list states in different

[jira] [Created] (FLINK-5863) Unify the serialization of queryable list states in different backends

2017-02-21 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-5863: --- Summary: Unify the serialization of queryable list states in different backends Key: FLINK-5863 URL: https://issues.apache.org/jira/browse/FLINK-5863 Project: Flink

[jira] [Closed] (FLINK-5036) Perform the grouping of keys in restoring instead of checkpointing

2017-02-15 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi closed FLINK-5036. --- Resolution: Invalid > Perform the grouping of keys in restoring instead of checkpointing >

[jira] [Closed] (FLINK-5023) Add get() method in State interface

2017-02-13 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi closed FLINK-5023. --- Resolution: Won't Fix The updates to the `State` interface will affect existing user code. We will

[jira] [Closed] (FLINK-5024) Add SimpleStateDescriptor to clarify the concepts

2017-02-13 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi closed FLINK-5024. --- Resolution: Invalid Now we refactor the state descriptors with the introduction of composited

[jira] [Created] (FLINK-5790) Use list types when ListStateDescriptor extends StateDescriptor

2017-02-13 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-5790: --- Summary: Use list types when ListStateDescriptor extends StateDescriptor Key: FLINK-5790 URL: https://issues.apache.org/jira/browse/FLINK-5790 Project: Flink

[jira] [Created] (FLINK-5738) Destroy created backend when task is canceled

2017-02-07 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-5738: --- Summary: Destroy created backend when task is canceled Key: FLINK-5738 URL: https://issues.apache.org/jira/browse/FLINK-5738 Project: Flink Issue Type: Bug

[jira] [Commented] (FLINK-5590) Create a proper internal state hierarchy

2017-01-21 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15833260#comment-15833260 ] Xiaogang Shi commented on FLINK-5590: - [~StephanEwen], do you have any ideas of the solution? I think

[jira] [Assigned] (FLINK-5544) Implement Internal Timer Service in RocksDB

2017-01-21 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi reassigned FLINK-5544: --- Assignee: Xiaogang Shi > Implement Internal Timer Service in RocksDB >

[jira] [Updated] (FLINK-5544) Implement Internal Timer Service in RocksDB

2017-01-17 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi updated FLINK-5544: Issue Type: New Feature (was: Bug) > Implement Internal Timer Service in RocksDB >

[jira] [Created] (FLINK-5544) Implement Internal Timer Service in RocksDB

2017-01-17 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-5544: --- Summary: Implement Internal Timer Service in RocksDB Key: FLINK-5544 URL: https://issues.apache.org/jira/browse/FLINK-5544 Project: Flink Issue Type: Bug

[jira] [Commented] (FLINK-5398) Exclude generated files in module flink-batch-connectors in license checking

2016-12-29 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786803#comment-15786803 ] Xiaogang Shi commented on FLINK-5398: - [~fhueske] Thanks for your explanation. It helps. > Exclude

[jira] [Updated] (FLINK-5400) Add accessor to folding states in RuntimeContext

2016-12-29 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi updated FLINK-5400: Description: Now {{RuntimeContext}} does not provide the accessors to folding states. Therefore

[jira] [Created] (FLINK-5400) Add accessor to folding states in RuntimeContext

2016-12-29 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-5400: --- Summary: Add accessor to folding states in RuntimeContext Key: FLINK-5400 URL: https://issues.apache.org/jira/browse/FLINK-5400 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-5398) Exclude generated files in module flink-batch-connectors in license checking

2016-12-28 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-5398: --- Summary: Exclude generated files in module flink-batch-connectors in license checking Key: FLINK-5398 URL: https://issues.apache.org/jira/browse/FLINK-5398 Project:

[jira] [Commented] (FLINK-5397) Fail to deserialize savepoints in v1.1 when there exist missing fields in class serialization descriptors

2016-12-28 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782811#comment-15782811 ] Xiaogang Shi commented on FLINK-5397: - The idea does work and is better. Very thanks for the quick fix

[jira] [Updated] (FLINK-5397) Fail to deserialize savepoints in v1.1 when there exist missing fields in class serialization descriptors

2016-12-28 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi updated FLINK-5397: Description: To restore from the savepoints in previous versions, Flink now keeps all classes

[jira] [Created] (FLINK-5397) Fail to deserialize savepoints in v1.1 when there exist missing fields in class serialization descriptors

2016-12-28 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-5397: --- Summary: Fail to deserialize savepoints in v1.1 when there exist missing fields in class serialization descriptors Key: FLINK-5397 URL:

[jira] [Commented] (FLINK-5214) Clean up checkpoint files when failing checkpoint operation on TM

2016-11-30 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15710512#comment-15710512 ] Xiaogang Shi commented on FLINK-5214: - I opened FLINK-5086 to report a similar problem, but I do not

[jira] [Commented] (FLINK-5090) Expose optionally detailed metrics about network queue lengths

2016-11-19 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15679596#comment-15679596 ] Xiaogang Shi commented on FLINK-5090: - In flink, the performance bottlenecks are usually caused by 1.

[jira] [Commented] (FLINK-5090) Expose optionally detailed metrics about network queue lengths

2016-11-17 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15675401#comment-15675401 ] Xiaogang Shi commented on FLINK-5090: - I suggest the metrics to be channel-wise. With these metrics,

[jira] [Created] (FLINK-5086) Clean dead snapshot files produced by the tasks failing to acknowledge checkpoints

2016-11-17 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-5086: --- Summary: Clean dead snapshot files produced by the tasks failing to acknowledge checkpoints Key: FLINK-5086 URL: https://issues.apache.org/jira/browse/FLINK-5086

[jira] [Comment Edited] (FLINK-5053) Incremental / lightweight snapshots for checkpoints

2016-11-17 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15673253#comment-15673253 ] Xiaogang Shi edited comment on FLINK-5053 at 11/17/16 9:41 AM: --- [~srichter]

[jira] [Commented] (FLINK-5053) Incremental / lightweight snapshots for checkpoints

2016-11-17 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15673253#comment-15673253 ] Xiaogang Shi commented on FLINK-5053: - [~srichter] Do you have a more detailed plan about incremental

[jira] [Commented] (FLINK-5085) Execute CheckpointCoodinator's state discard calls asynchronously

2016-11-16 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15673026#comment-15673026 ] Xiaogang Shi commented on FLINK-5085: - Great, this is what i thought of in recent days. Our states are

[jira] [Commented] (FLINK-5053) Incremental / lightweight snapshots for checkpoints

2016-11-14 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15663160#comment-15663160 ] Xiaogang Shi commented on FLINK-5053: - I think it's better to use {{checkpoint}} instead of {{backup}}

[jira] [Commented] (FLINK-5023) Add get() method in State interface

2016-11-09 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15653177#comment-15653177 ] Xiaogang Shi commented on FLINK-5023: - [~aljoscha] [~StephanEwen] I have updated the PR. Now, `State`

[jira] [Commented] (FLINK-5024) Add SimpleStateDescriptor to clarify the concepts

2016-11-09 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652901#comment-15652901 ] Xiaogang Shi commented on FLINK-5024: - I am very poor at English :( But i think "Simple" is more often

[jira] [Commented] (FLINK-5036) Perform the grouping of keys in restoring instead of checkpointing

2016-11-09 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15650462#comment-15650462 ] Xiaogang Shi commented on FLINK-5036: - Current implementation of both checkpointing and restoring

[jira] [Created] (FLINK-5036) Perform the grouping of keys in restoring instead of checkpointing

2016-11-08 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-5036: --- Summary: Perform the grouping of keys in restoring instead of checkpointing Key: FLINK-5036 URL: https://issues.apache.org/jira/browse/FLINK-5036 Project: Flink

[jira] [Comment Edited] (FLINK-5023) Add get() method in State interface

2016-11-08 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15649615#comment-15649615 ] Xiaogang Shi edited comment on FLINK-5023 at 11/9/16 2:52 AM: -- [~aljoscha] In

[jira] [Comment Edited] (FLINK-5023) Add get() method in State interface

2016-11-08 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15649615#comment-15649615 ] Xiaogang Shi edited comment on FLINK-5023 at 11/9/16 2:46 AM: -- @Aljoscha

[jira] [Comment Edited] (FLINK-5023) Add get() method in State interface

2016-11-08 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15649615#comment-15649615 ] Xiaogang Shi edited comment on FLINK-5023 at 11/9/16 2:45 AM: -- @Aljoscha

[jira] [Commented] (FLINK-5023) Add get() method in State interface

2016-11-08 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15649615#comment-15649615 ] Xiaogang Shi commented on FLINK-5023: - Aljoscha Krettek In that case, i think we should provide

[jira] [Assigned] (FLINK-5024) Add SimpleStateDescriptor to clarify the concepts

2016-11-07 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi reassigned FLINK-5024: --- Assignee: Xiaogang Shi > Add SimpleStateDescriptor to clarify the concepts >

[jira] [Assigned] (FLINK-5023) Add get() method in State interface

2016-11-07 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi reassigned FLINK-5023: --- Assignee: Xiaogang Shi > Add get() method in State interface >

[jira] [Commented] (FLINK-5023) Add get() method in State interface

2016-11-07 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15646299#comment-15646299 ] Xiaogang Shi commented on FLINK-5023: - I have opened a PR:

[jira] [Commented] (FLINK-5023) Add get() method in State interface

2016-11-07 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15643925#comment-15643925 ] Xiaogang Shi commented on FLINK-5023: - The only old method affected is the `value()` method in

[jira] [Comment Edited] (FLINK-4856) Add MapState for keyed streams

2016-11-06 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15642868#comment-15642868 ] Xiaogang Shi edited comment on FLINK-4856 at 11/7/16 2:29 AM: -- I have started

[jira] [Commented] (FLINK-4856) Add MapState for keyed streams

2016-11-06 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15642868#comment-15642868 ] Xiaogang Shi commented on FLINK-4856: - I have started the implementation of MapStates. But at prior to

[jira] [Created] (FLINK-5024) Add SimpleStateDescriptor to clarify the concepts

2016-11-06 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-5024: --- Summary: Add SimpleStateDescriptor to clarify the concepts Key: FLINK-5024 URL: https://issues.apache.org/jira/browse/FLINK-5024 Project: Flink Issue Type:

[jira] [Created] (FLINK-5023) Add get() method in State interface

2016-11-06 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-5023: --- Summary: Add get() method in State interface Key: FLINK-5023 URL: https://issues.apache.org/jira/browse/FLINK-5023 Project: Flink Issue Type: Improvement

[jira] [Assigned] (FLINK-4856) Add MapState for keyed streams

2016-10-19 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi reassigned FLINK-4856: --- Assignee: Xiaogang Shi > Add MapState for keyed streams > -- > >

[jira] [Created] (FLINK-4856) Add MapState for keyed streams

2016-10-18 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-4856: --- Summary: Add MapState for keyed streams Key: FLINK-4856 URL: https://issues.apache.org/jira/browse/FLINK-4856 Project: Flink Issue Type: New Feature

[jira] [Closed] (FLINK-4448) Use Listeners to monitor execution status

2016-08-23 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi closed FLINK-4448. --- Resolution: Fixed The functionality will be implemented in Flink-4457

[jira] [Commented] (FLINK-4408) Submit Job and setup ExecutionGraph

2016-08-23 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15434052#comment-15434052 ] Xiaogang Shi commented on FLINK-4408: - I think it's okay to pull the logics about leadership out of

[jira] [Created] (FLINK-4448) Use Listeners to monitor execution status

2016-08-22 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-4448: --- Summary: Use Listeners to monitor execution status Key: FLINK-4448 URL: https://issues.apache.org/jira/browse/FLINK-4448 Project: Flink Issue Type: Sub-task

[jira] [Updated] (FLINK-4408) Submit Job and setup ExecutionGraph

2016-08-17 Thread Xiaogang Shi (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaogang Shi updated FLINK-4408: Summary: Submit Job and setup ExecutionGraph (was: JobSubmission) > Submit Job and setup

[jira] [Created] (FLINK-4408) JobSubmission

2016-08-17 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-4408: --- Summary: JobSubmission Key: FLINK-4408 URL: https://issues.apache.org/jira/browse/FLINK-4408 Project: Flink Issue Type: Sub-task Components: Cluster

[jira] [Created] (FLINK-4400) Leadership Election among JobManagers

2016-08-16 Thread Xiaogang Shi (JIRA)
Xiaogang Shi created FLINK-4400: --- Summary: Leadership Election among JobManagers Key: FLINK-4400 URL: https://issues.apache.org/jira/browse/FLINK-4400 Project: Flink Issue Type: Sub-task