[jira] [Commented] (FLINK-5510) Replace Scala Future with FlinkFuture in QueryableStateClient

2017-10-09 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16196560#comment-16196560 ] Yun Tang commented on FLINK-5510: - According to [FLINK-725 (Remove Flink Futures or back them by

[jira] [Created] (FLINK-9328) RocksDBStateBackend might use PlaceholderStreamStateHandle to restor due to StateBackendTestBase class not register snapshots in some UTs

2018-05-10 Thread Yun Tang (JIRA)
Yun Tang created FLINK-9328: --- Summary: RocksDBStateBackend might use PlaceholderStreamStateHandle to restor due to StateBackendTestBase class not register snapshots in some UTs Key: FLINK-9328 URL:

[jira] [Commented] (FLINK-9598) [Checkpoints] The config Minimum Pause Between Checkpoints doesn't work when there's a checkpoint failure

2018-07-16 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545318#comment-16545318 ] Yun Tang commented on FLINK-9598: - Hi [~premsantosh], the "Minimum Pause Between Checkpoints" is actually

[jira] [Assigned] (FLINK-9598) [Checkpoints] The config Minimum Pause Between Checkpoints doesn't work when there's a checkpoint failure

2018-07-16 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-9598: --- Assignee: Yun Tang > [Checkpoints] The config Minimum Pause Between Checkpoints doesn't work when

[jira] [Updated] (FLINK-10267) [State] Fix arbitrary iterator access on RocksDBMapIterator

2018-08-30 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-10267: - Description: Currently, RocksDBMapIterator would load 128 entries into local cacheEntries every time

[jira] [Created] (FLINK-10267) [State] Fix arbitrary iterator access on RocksDBMapIterator

2018-08-30 Thread Yun Tang (JIRA)
Yun Tang created FLINK-10267: Summary: [State] Fix arbitrary iterator access on RocksDBMapIterator Key: FLINK-10267 URL: https://issues.apache.org/jira/browse/FLINK-10267 Project: Flink Issue

[jira] [Created] (FLINK-9758) ContinuousFileProcessingTest failed due to not setting runtimeContext

2018-07-04 Thread Yun Tang (JIRA)
Yun Tang created FLINK-9758: --- Summary: ContinuousFileProcessingTest failed due to not setting runtimeContext Key: FLINK-9758 URL: https://issues.apache.org/jira/browse/FLINK-9758 Project: Flink

[jira] [Commented] (FLINK-4387) Instability in KvStateClientTest.testClientServerIntegration()

2018-07-10 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16538561#comment-16538561 ] Yun Tang commented on FLINK-4387: - [~NicoK] Has this issue already been fixed? I noticed you said this

[jira] [Assigned] (FLINK-10712) RestartPipelinedRegionStrategy does not restore state

2018-10-31 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-10712: Assignee: Yun Tang > RestartPipelinedRegionStrategy does not restore state >

[jira] [Commented] (FLINK-10712) RestartPipelinedRegionStrategy does not restore state

2018-10-30 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668964#comment-16668964 ] Yun Tang commented on FLINK-10712: -- We have refactored _FailoverRegion.java_ to support fail-over with

[jira] [Comment Edited] (FLINK-10712) RestartPipelinedRegionStrategy does not restore state

2018-10-30 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668964#comment-16668964 ] Yun Tang edited comment on FLINK-10712 at 10/30/18 4:18 PM: We have already

[jira] [Comment Edited] (FLINK-10855) CheckpointCoordinator does not delete checkpoint directory of late/failed checkpoints

2018-11-13 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686065#comment-16686065 ] Yun Tang edited comment on FLINK-10855 at 11/14/18 3:54 AM:

[jira] [Commented] (FLINK-10855) CheckpointCoordinator does not delete checkpoint directory of late/failed checkpoints

2018-11-13 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686065#comment-16686065 ] Yun Tang commented on FLINK-10855: -- [~till.rohrmann], as far as I know, the checkpoint directory is

[jira] [Assigned] (FLINK-11363) Check and remove TaskManagerConfigurationTest

2019-01-16 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-11363: Assignee: Yun Tang > Check and remove TaskManagerConfigurationTest >

[jira] [Assigned] (FLINK-11366) Check and port TaskManagerMetricsTest to new code base if necessary

2019-01-16 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-11366: Assignee: Yun Tang > Check and port TaskManagerMetricsTest to new code base if necessary >

[jira] [Assigned] (FLINK-11368) Check and port TaskManagerStartupTest to new code base if necessary

2019-01-17 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-11368: Assignee: Yun Tang > Check and port TaskManagerStartupTest to new code base if necessary >

[jira] [Assigned] (FLINK-11333) First-class support for Protobuf types with evolvable schema

2019-01-15 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-11333: Assignee: Yun Tang > First-class support for Protobuf types with evolvable schema >

[jira] [Commented] (FLINK-9920) BucketingSinkFaultToleranceITCase fails on travis

2019-01-21 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16748419#comment-16748419 ] Yun Tang commented on FLINK-9920: - Another instance: 

[jira] [Commented] (FLINK-11380) YarnFlinkResourceManagerTest test case crashed

2019-01-21 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16748350#comment-16748350 ] Yun Tang commented on FLINK-11380: -- Another instance https://api.travis-ci.org/v3/job/481937341/log.txt

[jira] [Commented] (FLINK-10724) Refactor failure handling in check point coordinator

2018-12-10 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16715194#comment-16715194 ] Yun Tang commented on FLINK-10724: -- [~yanghua] I think CheckpointSubsumed should not be counted as

[jira] [Commented] (FLINK-11138) RocksDB's wal seems to be useless in rocksdb-state-backend that reduce the disk's overhead

2018-12-12 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719713#comment-16719713 ] Yun Tang commented on FLINK-11138: -- Yes, you are right. But we already disable the WAL of rocksDB. You

[jira] [Commented] (FLINK-10912) Configurable RocksDBStateBackend options factory

2018-12-24 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16728304#comment-16728304 ] Yun Tang commented on FLINK-10912: -- [~thw] I would like to take this issue if you don't mind. We

[jira] [Commented] (FLINK-11212) operator metric key can repalce operator name by operator id

2018-12-24 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16728299#comment-16728299 ] Yun Tang commented on FLINK-11212: -- >From my point of view, the operator id is not so readable, which

[jira] [Commented] (FLINK-11171) Unexpected timestamp deserialization failure in RocksDB state backend

2018-12-17 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722952#comment-16722952 ] Yun Tang commented on FLINK-11171: -- [~sayatez] I think the 2nd problem should have no relationship with

[jira] [Commented] (FLINK-10930) Refactor checkpoint directory layout

2018-12-23 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16728133#comment-16728133 ] Yun Tang commented on FLINK-10930: -- I plan to close this issue, and we should move to

[jira] [Closed] (FLINK-10930) Refactor checkpoint directory layout

2018-12-23 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang closed FLINK-10930. Resolution: Won't Fix > Refactor checkpoint directory layout > > >

[jira] [Assigned] (FLINK-10912) Configurable RocksDBStateBackend options factory

2018-12-24 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-10912: Assignee: Yun Tang > Configurable RocksDBStateBackend options factory >

[jira] [Commented] (FLINK-10930) Refactor checkpoint directory layout

2018-12-04 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708978#comment-16708978 ] Yun Tang commented on FLINK-10930: -- Speaking of eager and proper discarding useless state, this is

[jira] [Updated] (FLINK-11107) [state] Avoid memory stateBackend to create arbitrary folders under HA path when no checkpoint path configured

2018-12-09 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-11107: - Description: Currently, memory state-backend would create a folder named with random UUID under HA

[jira] [Commented] (FLINK-11107) [state] Avoid memory stateBackend to create arbitrary folders under HA path when no checkpoint path configured

2018-12-09 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714033#comment-16714033 ] Yun Tang commented on FLINK-11107: -- @ [~StephanEwen] , as you commented in code below: {noformat} // to

[jira] [Commented] (FLINK-10930) Refactor checkpoint directory layout

2018-11-29 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704286#comment-16704286 ] Yun Tang commented on FLINK-10930: -- [~StephanEwen] I think your opinion to refactor the mechanism of

[jira] [Comment Edited] (FLINK-9043) Introduce a friendly way to resume the job from externalized checkpoints automatically

2018-11-19 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692610#comment-16692610 ] Yun Tang edited comment on FLINK-9043 at 11/20/18 3:55 AM: --- I quite agree with

[jira] [Commented] (FLINK-9043) Introduce a friendly way to resume the job from externalized checkpoints automatically

2018-11-19 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692610#comment-16692610 ] Yun Tang commented on FLINK-9043: - I quite agreed with [~sihuazhou]'s idea, user just need to provide the

[jira] [Commented] (FLINK-10930) Refactor checkpoint directory layout

2018-11-25 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16698560#comment-16698560 ] Yun Tang commented on FLINK-10930: -- [~StephanEwen] I know this suggested changes would impact your

[jira] [Commented] (FLINK-10930) Refactor checkpoint directory layout

2018-11-27 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16701359#comment-16701359 ] Yun Tang commented on FLINK-10930: -- [~StephanEwen] I try to answer your points of view: * _The error

[jira] [Assigned] (FLINK-10954) Hardlink from files of previous local stored state might cross devices

2018-11-20 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-10954: Assignee: Yun Tang > Hardlink from files of previous local stored state might cross devices >

[jira] [Commented] (FLINK-10954) Hardlink from files of previous local stored state might cross devices

2018-11-20 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693531#comment-16693531 ] Yun Tang commented on FLINK-10954: -- First of all, I plan to handle the exception after calling

[jira] [Created] (FLINK-10954) Hardlink from files of previous local stored state might cross devices

2018-11-20 Thread Yun Tang (JIRA)
Yun Tang created FLINK-10954: Summary: Hardlink from files of previous local stored state might cross devices Key: FLINK-10954 URL: https://issues.apache.org/jira/browse/FLINK-10954 Project: Flink

[jira] [Created] (FLINK-10930) Refactor checkpoint directory layout

2018-11-19 Thread Yun Tang (JIRA)
Yun Tang created FLINK-10930: Summary: Refactor checkpoint directory layout Key: FLINK-10930 URL: https://issues.apache.org/jira/browse/FLINK-10930 Project: Flink Issue Type: Improvement

[jira] [Assigned] (FLINK-11292) Document how to use the new CompositeTypeSerializerSnapshot

2019-01-11 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-11292: Assignee: Yun Tang > Document how to use the new CompositeTypeSerializerSnapshot >

[jira] [Created] (FLINK-11313) [checkpoint] Introduce LZ4 compression for keyed state in full checkpoints and savepoints

2019-01-14 Thread Yun Tang (JIRA)
Yun Tang created FLINK-11313: Summary: [checkpoint] Introduce LZ4 compression for keyed state in full checkpoints and savepoints Key: FLINK-11313 URL: https://issues.apache.org/jira/browse/FLINK-11313

[jira] [Created] (FLINK-11107) [state] Avoid memory stateBackend to create arbitrary folders under HA path when no checkpoint path configured

2018-12-09 Thread Yun Tang (JIRA)
Yun Tang created FLINK-11107: Summary: [state] Avoid memory stateBackend to create arbitrary folders under HA path when no checkpoint path configured Key: FLINK-11107 URL:

[jira] [Commented] (FLINK-10930) Refactor checkpoint directory layout

2018-12-09 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16713903#comment-16713903 ] Yun Tang commented on FLINK-10930: -- [~StephanEwen] The ideas of cleaning up exclusive and shared state

[jira] [Commented] (FLINK-11437) EventTimeWindowCheckpointingITCase#testTumblingTimeWindowWithKVStateMaxMaxParallelism caused CancellationException

2019-01-28 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16754159#comment-16754159 ] Yun Tang commented on FLINK-11437: --  From the detailed message, I think this is because the task

[jira] [Commented] (FLINK-12080) Add timestamp property to savepoint

2019-04-02 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16807443#comment-16807443 ] Yun Tang commented on FLINK-12080: -- As discussed with Huang in user

[jira] [Commented] (FLINK-12096) Flink counter state getting stuck on usin

2019-04-03 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808647#comment-16808647 ] Yun Tang commented on FLINK-12096: -- [~yogesh_n] Would you please provide more information such as

[jira] [Closed] (FLINK-11948) When kafka sink parallelism

2019-03-28 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang closed FLINK-11948. Resolution: Duplicate > When kafka sink parallelism unbalance >

[jira] [Commented] (FLINK-12058) Cancel checkpoint operations belonging to a discarded/aborted checkpoint

2019-03-28 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804581#comment-16804581 ] Yun Tang commented on FLINK-12058: -- Hi [~till.rohrmann], is this a duplicate of FLINK-8871 ? > Cancel

[jira] [Commented] (FLINK-12043) Null value check in array serializers classes

2019-03-29 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804652#comment-16804652 ] Yun Tang commented on FLINK-12043: -- hi [~quan], what's your purpose to report this Jira: * Complain

[jira] [Commented] (FLINK-11789) Checkpoint directories are not cleaned up after job termination

2019-03-01 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16782314#comment-16782314 ] Yun Tang commented on FLINK-11789: -- [~shengjk1] When job fails, we should also consider the

[jira] [Commented] (FLINK-11700) mvn package field

2019-02-20 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773722#comment-16773722 ] Yun Tang commented on FLINK-11700: -- Does the class {{org/apache/commons/lang3/SystemUtils}} existed in

[jira] [Updated] (FLINK-11691) Introduce ClassLoader in the methods of StateBackendFactory and ConfigurableStateBackend

2019-02-20 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-11691: - Description: Currently, interface {{ConfigurableStateBackend}} has method definition below: {code:java}

[jira] [Created] (FLINK-11691) Introduce ClassLoader in the methods of StateBackendFactory and ConfigurableStateBackend

2019-02-20 Thread Yun Tang (JIRA)
Yun Tang created FLINK-11691: Summary: Introduce ClassLoader in the methods of StateBackendFactory and ConfigurableStateBackend Key: FLINK-11691 URL: https://issues.apache.org/jira/browse/FLINK-11691

[jira] [Updated] (FLINK-11691) Introduce ClassLoader in the methods of StateBackendFactory and ConfigurableStateBackend

2019-02-20 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-11691: - Description: Currently, interface {{ConfigurableStateBackend}} has method definition below:

[jira] [Assigned] (FLINK-11691) Introduce ClassLoader in the methods of StateBackendFactory and ConfigurableStateBackend

2019-02-20 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-11691: Assignee: Yun Tang > Introduce ClassLoader in the methods of StateBackendFactory and >

[jira] [Updated] (FLINK-11691) Introduce ClassLoader in the methods of StateBackendFactory and ConfigurableStateBackend

2019-02-20 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-11691: - Description: Currently, interface {{ConfigurableStateBackend}} has method definition below: {code:java}

[jira] [Created] (FLINK-11696) [checkpoint] Avoid to send mkdir requests to DFS from task side

2019-02-20 Thread Yun Tang (JIRA)
Yun Tang created FLINK-11696: Summary: [checkpoint] Avoid to send mkdir requests to DFS from task side Key: FLINK-11696 URL: https://issues.apache.org/jira/browse/FLINK-11696 Project: Flink

[jira] [Updated] (FLINK-11695) [checkpoint] Make sharedStateDir could create sub-directories to avoid MaxDirectoryItemsExceededException

2019-02-20 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-11695: - Description: We meet this annoying problem many times when the {{sharedStateDir}} in checkpoint path

[jira] [Created] (FLINK-11695) [checkpoint] Make sharedStateDir could create sub-directories to avoid MaxDirectoryItemsExceededException

2019-02-20 Thread Yun Tang (JIRA)
Yun Tang created FLINK-11695: Summary: [checkpoint] Make sharedStateDir could create sub-directories to avoid MaxDirectoryItemsExceededException Key: FLINK-11695 URL: https://issues.apache.org/jira/browse/FLINK-11695

[jira] [Created] (FLINK-11829) [checkpoint] FsCheckpointStateOutputStream might store state in files even state size below fileStateThreshold

2019-03-05 Thread Yun Tang (JIRA)
Yun Tang created FLINK-11829: Summary: [checkpoint] FsCheckpointStateOutputStream might store state in files even state size below fileStateThreshold Key: FLINK-11829 URL:

[jira] [Commented] (FLINK-11835) ZooKeeperLeaderElectionITCase#testJobExecutionOnClusterWithLeaderChange failed

2019-03-14 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16792421#comment-16792421 ] Yun Tang commented on FLINK-11835: -- Another instance

[jira] [Commented] (FLINK-11926) The Description of Assigners allowing a fixed amount of lateness is not correct

2019-03-18 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794769#comment-16794769 ] Yun Tang commented on FLINK-11926: -- Yes, you could refer to the doc

[jira] [Commented] (FLINK-11948) When kafka sink parallelism

2019-03-18 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794767#comment-16794767 ] Yun Tang commented on FLINK-11948: -- I think this is what {{FlinkFixedPartitioner}} wants to achieve.

[jira] [Comment Edited] (FLINK-11948) When kafka sink parallelism

2019-03-18 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794767#comment-16794767 ] Yun Tang edited comment on FLINK-11948 at 3/18/19 6:43 AM: --- I think this is

[jira] [Commented] (FLINK-11874) Split CheckpointStorage interface to distinguish JM and TM side

2019-03-11 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790221#comment-16790221 ] Yun Tang commented on FLINK-11874: -- [~yanghua] Thanks for your suggestion. I thought I had given the

[jira] [Updated] (FLINK-11874) Split CheckpointStorage interface to distinguish JM and TM side

2019-03-11 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-11874: - Summary: Split CheckpointStorage interface to distinguish JM and TM side (was: [checkpoint] Split

[jira] [Commented] (FLINK-11107) [state] Avoid memory stateBackend to create arbitrary folders under HA path when no checkpoint path configured

2019-03-13 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791765#comment-16791765 ] Yun Tang commented on FLINK-11107: -- User also meet this problem and report it in [chinese mail

[jira] [Updated] (FLINK-11868) [filesystems] Introduce listStatusIterator API to file system

2019-03-09 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-11868: - Description: >From existed experience, we know {{listStatus}} is expensive for many >distributed file

[jira] [Created] (FLINK-11868) [filesystems] Introduce listStatusIterator API to file system

2019-03-09 Thread Yun Tang (JIRA)
Yun Tang created FLINK-11868: Summary: [filesystems] Introduce listStatusIterator API to file system Key: FLINK-11868 URL: https://issues.apache.org/jira/browse/FLINK-11868 Project: Flink Issue

[jira] [Created] (FLINK-11869) [checkpoint] Make buffer size in checkpoint stream factory configurable

2019-03-09 Thread Yun Tang (JIRA)
Yun Tang created FLINK-11869: Summary: [checkpoint] Make buffer size in checkpoint stream factory configurable Key: FLINK-11869 URL: https://issues.apache.org/jira/browse/FLINK-11869 Project: Flink

[jira] [Commented] (FLINK-11962) Huge Flink Checkpoint Size When Using "Group By" Clause With Table SQL

2019-03-19 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16795803#comment-16795803 ] Yun Tang commented on FLINK-11962: -- [~GerogeRyd] The different checkpoint sizes reflect different

[jira] [Commented] (FLINK-11948) When kafka sink parallelism

2019-03-18 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794878#comment-16794878 ] Yun Tang commented on FLINK-11948: -- [~bf30075] I think the name {{FlinkFixedPartitioner}} should

[jira] [Commented] (FLINK-11860) Remove all the usage of deprecated unit-provided memory options in docs and scripts

2019-03-07 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787629#comment-16787629 ] Yun Tang commented on FLINK-11860: -- This issue is inspired by user mail

[jira] [Created] (FLINK-11860) Remove all the usage of deprecated unit-provided memory options in docs and scripts

2019-03-07 Thread Yun Tang (JIRA)
Yun Tang created FLINK-11860: Summary: Remove all the usage of deprecated unit-provided memory options in docs and scripts Key: FLINK-11860 URL: https://issues.apache.org/jira/browse/FLINK-11860 Project:

[jira] [Assigned] (FLINK-11860) Remove all the usage of deprecated unit-provided memory options in docs and scripts

2019-03-07 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-11860: Assignee: Yun Tang > Remove all the usage of deprecated unit-provided memory options in docs and

[jira] [Commented] (FLINK-11926) The Description of Assigners allowing a fixed amount of lateness is not correct

2019-03-15 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793479#comment-16793479 ] Yun Tang commented on FLINK-11926: -- [~chaiyq] lateness means {{time-of-event}} minus

[jira] [Closed] (FLINK-11926) The Description of Assigners allowing a fixed amount of lateness is not correct

2019-03-15 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang closed FLINK-11926. Resolution: Not A Problem > The Description of Assigners allowing a fixed amount of lateness is not >

[jira] [Created] (FLINK-11874) [checkpoint] Split CheckpointStorage interface to distinguish JM and TM side

2019-03-11 Thread Yun Tang (JIRA)
Yun Tang created FLINK-11874: Summary: [checkpoint] Split CheckpointStorage interface to distinguish JM and TM side Key: FLINK-11874 URL: https://issues.apache.org/jira/browse/FLINK-11874 Project: Flink

[jira] [Commented] (FLINK-8871) Checkpoint cancellation is not propagated to stop checkpointing threads on the task manager

2019-03-11 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16789755#comment-16789755 ] Yun Tang commented on FLINK-8871: - We already introduce {{notifyCheckpointAbort}} which is like current

[jira] [Commented] (FLINK-11536) cannot start standalone cluster - failed to bind to /0.0.0.0:6123

2019-02-06 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16761859#comment-16761859 ] Yun Tang commented on FLINK-11536: -- hi [~hw271] I think *user*@flink.apache.org (

[jira] [Commented] (FLINK-11641) flink seems to be stuck when deploying new job (local environment)

2019-02-17 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770397#comment-16770397 ] Yun Tang commented on FLINK-11641: -- I think this is because your local mac does not know your machine's

[jira] [Created] (FLINK-11618) [state] Refactor operator state repartition mechanism

2019-02-14 Thread Yun Tang (JIRA)
Yun Tang created FLINK-11618: Summary: [state] Refactor operator state repartition mechanism Key: FLINK-11618 URL: https://issues.apache.org/jira/browse/FLINK-11618 Project: Flink Issue Type:

[jira] [Updated] (FLINK-11618) [state] Refactor operator state repartition mechanism

2019-02-14 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-11618: - Description: Currently we have state assignment strategy of operator state below: * When parallelism

[jira] [Commented] (FLINK-11662) job restart when CheckpointCoordinator drop checkpointDirectory as a whole

2019-02-19 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16772694#comment-16772694 ] Yun Tang commented on FLINK-11662: -- [~framst] we already have some discussion about this problem that

[jira] [Commented] (FLINK-11333) First-class support for Protobuf types with evolvable schema

2019-01-29 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755185#comment-16755185 ] Yun Tang commented on FLINK-11333: -- I planed to use avro-protobuf to extract Protobuf message's schema

[jira] [Commented] (FLINK-11427) Protobuf parquet writer implementation

2019-01-25 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752126#comment-16752126 ] Yun Tang commented on FLINK-11427: -- [~zjuguanghu] I just have a curious question which is a bit

[jira] [Commented] (FLINK-9920) BucketingSinkFaultToleranceITCase fails on travis

2019-01-31 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-9920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757409#comment-16757409 ] Yun Tang commented on FLINK-9920: - With more detailed messages, I believe this bug happened due to the

[jira] [Comment Edited] (FLINK-11107) [state] Avoid memory stateBackend to create arbitrary folders under HA path when no checkpoint path configured

2019-04-12 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815949#comment-16815949 ] Yun Tang edited comment on FLINK-11107 at 4/12/19 8:43 AM: --- [~ykt836] This is

[jira] [Commented] (FLINK-11107) [state] Avoid memory stateBackend to create arbitrary folders under HA path when no checkpoint path configured

2019-04-11 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815949#comment-16815949 ] Yun Tang commented on FLINK-11107: -- [~ykt836] This is the issue which would created a lot of useless

[jira] [Updated] (FLINK-11107) [state] Avoid memory stateBackend to create arbitrary folders under HA path when no checkpoint path configured

2019-04-14 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang updated FLINK-11107: - Priority: Blocker (was: Major) Fix Version/s: (was: 1.6.5) (was:

[jira] [Assigned] (FLINK-6984) Do not recover CompletedCheckpointStore on every restore

2019-04-15 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-6984: --- Assignee: Yun Tang > Do not recover CompletedCheckpointStore on every restore >

[jira] [Commented] (FLINK-10912) Configurable RocksDBStateBackend options factory

2019-03-15 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793662#comment-16793662 ] Yun Tang commented on FLINK-10912: -- [~ZhenqiuHuang] If you really want to apply this feature to

[jira] [Comment Edited] (FLINK-10912) Configurable RocksDBStateBackend options factory

2019-03-15 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793662#comment-16793662 ] Yun Tang edited comment on FLINK-10912 at 3/15/19 2:22 PM: --- [~ZhenqiuHuang] If

[jira] [Commented] (FLINK-8871) Checkpoint cancellation is not propagated to stop checkpointing threads on the task manager

2019-05-25 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848149#comment-16848149 ] Yun Tang commented on FLINK-8871: - Thanks for [~srichter]'s comment. I agree we should fix FLINK-11662

[jira] [Commented] (FLINK-12546) Base Docker images on `library/flink`

2019-05-28 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16849904#comment-16849904 ] Yun Tang commented on FLINK-12546: -- Thanks for [~till.rohrmann]'s explanation, I have another question

[jira] [Commented] (FLINK-12820) Support ignoring null fields when writing to Cassandra

2019-06-13 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862843#comment-16862843 ] Yun Tang commented on FLINK-12820: -- Would you please help to clarify why current solution is impossible

[jira] [Commented] (FLINK-12820) Support ignoring null fields when writing to Cassandra

2019-06-16 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865105#comment-16865105 ] Yun Tang commented on FLINK-12820: -- [~ozancicek] Sorry for late reply and thanks for your kind

[jira] [Commented] (FLINK-12859) ClassNotFoundException during MapRFs init

2019-06-15 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16864791#comment-16864791 ] Yun Tang commented on FLINK-12859: -- [~aitozi] I am afraid that the attached image has been broken to

[jira] [Assigned] (FLINK-6931) Support custom compression formats for checkpoints (+Upgrade/Compatibility)

2019-06-11 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-6931: --- Assignee: Yun Tang > Support custom compression formats for checkpoints (+Upgrade/Compatibility) >

[jira] [Commented] (FLINK-6931) Support custom compression formats for checkpoints (+Upgrade/Compatibility)

2019-06-11 Thread Yun Tang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860668#comment-16860668 ] Yun Tang commented on FLINK-6931: - I think https://github.com/apache/flink/pull/7819 should already cover

  1   2   3   4   5   6   7   8   9   10   >