[jira] [Commented] (FLINK-13100) Fix the unexpected IOException during FileBufferReader#nextBuffer

2019-07-08 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-13100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880465#comment-16880465 ] zhijiang commented on FLINK-13100: -- Yes, the previous SpilledSubpartitionView really has the deadlock

[jira] [Commented] (FLINK-11082) Fix the calculation of backlog in PipelinedSubpartition

2019-07-08 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880427#comment-16880427 ] zhijiang commented on FLINK-11082: -- [~pnowojski], if the requested floating buffers are not used in

[jira] [Created] (FLINK-13141) Remove getBufferSize method from BufferPoolFactory

2019-07-08 Thread zhijiang (JIRA)
zhijiang created FLINK-13141: Summary: Remove getBufferSize method from BufferPoolFactory Key: FLINK-13141 URL: https://issues.apache.org/jira/browse/FLINK-13141 Project: Flink Issue Type:

[jira] [Issue Comment Deleted] (FLINK-12852) Deadlock occurs when requiring exclusive buffer for RemoteInputChannel

2019-07-07 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12852: - Comment: was deleted (was: I thought of another option to keep the dynamic local pool as now and only

[jira] [Commented] (FLINK-12852) Deadlock occurs when requiring exclusive buffer for RemoteInputChannel

2019-07-07 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16879997#comment-16879997 ] zhijiang commented on FLINK-12852: -- I thought of another option to keep the dynamic local pool as now

[jira] [Comment Edited] (FLINK-12852) Deadlock occurs when requiring exclusive buffer for RemoteInputChannel

2019-07-05 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16879396#comment-16879396 ] zhijiang edited comment on FLINK-12852 at 7/5/19 3:58 PM: -- Thanks for above

[jira] [Commented] (FLINK-12852) Deadlock occurs when requiring exclusive buffer for RemoteInputChannel

2019-07-05 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16879396#comment-16879396 ] zhijiang commented on FLINK-12852: -- Thanks for above detail conclusions [~StephanEwen]! I agree that

[jira] [Created] (FLINK-13126) Construct special test/benchmark to verify the backlog effect

2019-07-05 Thread zhijiang (JIRA)
zhijiang created FLINK-13126: Summary: Construct special test/benchmark to verify the backlog effect Key: FLINK-13126 URL: https://issues.apache.org/jira/browse/FLINK-13126 Project: Flink Issue

[jira] [Updated] (FLINK-11082) Fix the calculation of backlog in PipelinedSubpartition

2019-07-05 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-11082: - Affects Version/s: (was: 1.6.3) (was: 1.8.0) > Fix the calculation of

[jira] [Updated] (FLINK-11082) Fix the calculation of backlog in PipelinedSubpartition

2019-07-05 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-11082: - Description: The backlog of subpartition should indicate how many buffers are consumable, then the

[jira] [Updated] (FLINK-11082) Fix the calculation of backlog in PipelinedSubpartition

2019-07-05 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-11082: - Summary: Fix the calculation of backlog in PipelinedSubpartition (was: Increase backlog only if it is

[jira] [Commented] (FLINK-12852) Deadlock occurs when requiring exclusive buffer for RemoteInputChannel

2019-07-04 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16878692#comment-16878692 ] zhijiang commented on FLINK-12852: -- Thanks for concerning this issue [~aitozi] We already have the

[jira] [Commented] (FLINK-12852) Deadlock occurs when requiring exclusive buffer for RemoteInputChannel

2019-07-04 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16878518#comment-16878518 ] zhijiang commented on FLINK-12852: -- We have two directions in general: buffers sharing between tasks or

[jira] [Commented] (FLINK-12852) Deadlock occurs when requiring exclusive buffer for RemoteInputChannel

2019-07-04 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16878516#comment-16878516 ] zhijiang commented on FLINK-12852: -- [~pnowojski] "Network resource matching in slot" refers to the

[jira] [Commented] (FLINK-13100) Fix the unexpected IOException during FileBufferReader#nextBuffer

2019-07-04 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-13100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16878513#comment-16878513 ] zhijiang commented on FLINK-13100: -- What do you think [~pnowojski] [~StephanEwen] > Fix the unexpected

[jira] [Created] (FLINK-13100) Fix the unexpected IOException during FileBufferReader#nextBuffer

2019-07-04 Thread zhijiang (JIRA)
zhijiang created FLINK-13100: Summary: Fix the unexpected IOException during FileBufferReader#nextBuffer Key: FLINK-13100 URL: https://issues.apache.org/jira/browse/FLINK-13100 Project: Flink

[jira] [Resolved] (FLINK-12738) Remove abstract getPageSize method from InputGate

2019-07-04 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang resolved FLINK-12738. -- Resolution: Fixed Fixed in b6563e385263beb556ce19855ca6dfdbb7f6c853 > Remove abstract getPageSize

[jira] [Comment Edited] (FLINK-12852) Deadlock occurs when requiring exclusive buffer for RemoteInputChannel

2019-07-03 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16878289#comment-16878289 ] zhijiang edited comment on FLINK-12852 at 7/4/19 3:27 AM: -- Network resource

[jira] [Commented] (FLINK-12852) Deadlock occurs when requiring exclusive buffer for RemoteInputChannel

2019-07-03 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16878289#comment-16878289 ] zhijiang commented on FLINK-12852: -- Network resource matching in slot has many unsetting issues which

[jira] [Commented] (FLINK-12852) Deadlock occurs when requiring exclusive buffer for RemoteInputChannel

2019-07-03 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16877737#comment-16877737 ] zhijiang commented on FLINK-12852: -- It is hardly to say whether we could get performance benefits via 

[jira] [Commented] (FLINK-12852) Deadlock occurs when requiring exclusive buffer for RemoteInputChannel

2019-07-03 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16877616#comment-16877616 ] zhijiang commented on FLINK-12852: -- Hey [~gaoyunhaii], considering the second point for long run, I

[jira] [Commented] (FLINK-12852) Deadlock occurs when requiring exclusive buffer for RemoteInputChannel

2019-07-03 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16877605#comment-16877605 ] zhijiang commented on FLINK-12852: -- The spill way could solve the deadlock issue in function, which is

[jira] [Commented] (FLINK-12852) Deadlock occurs when requiring exclusive buffer for RemoteInputChannel

2019-07-02 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16877073#comment-16877073 ] zhijiang commented on FLINK-12852: -- [~StephanEwen] thanks for concerning this issue and sharing the

[jira] [Updated] (FLINK-12882) Remove ExecutionAttemptID field from ShuffleEnvironment#createResultPartitionWriters

2019-06-26 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12882: - Description: The ResultPartitionID could be got directly from ResultPartitionDeploymentDescriptor, so

[jira] [Updated] (FLINK-12882) Remove ExecutionAttemptID argument from ResultPartitionFactory#create

2019-06-26 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12882: - Summary: Remove ExecutionAttemptID argument from ResultPartitionFactory#create (was: Remove

[jira] [Created] (FLINK-13010) Refactor the process of SchedulerNG#requestPartitionState

2019-06-26 Thread zhijiang (JIRA)
zhijiang created FLINK-13010: Summary: Refactor the process of SchedulerNG#requestPartitionState Key: FLINK-13010 URL: https://issues.apache.org/jira/browse/FLINK-13010 Project: Flink Issue

[jira] [Updated] (FLINK-12284) InputBufferPoolUsage is incorrect in credit-based network control flow

2019-06-26 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12284: - Component/s: Runtime / Metrics > InputBufferPoolUsage is incorrect in credit-based network control flow

[jira] [Comment Edited] (FLINK-12889) Job keeps in FAILING state

2019-06-25 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16872933#comment-16872933 ] zhijiang edited comment on FLINK-12889 at 6/26/19 5:33 AM: --- My previous

[jira] [Comment Edited] (FLINK-12889) Job keeps in FAILING state

2019-06-25 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16872933#comment-16872933 ] zhijiang edited comment on FLINK-12889 at 6/26/19 5:32 AM: --- My previous

[jira] [Updated] (FLINK-12889) Job keeps in FAILING state

2019-06-25 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12889: - Priority: Minor (was: Blocker) > Job keeps in FAILING state > -- > >

[jira] [Updated] (FLINK-12889) Job keeps in FAILING state

2019-06-25 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12889: - Affects Version/s: (was: 1.7.2) > Job keeps in FAILING state > -- > >

[jira] [Commented] (FLINK-12889) Job keeps in FAILING state

2019-06-25 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16872933#comment-16872933 ] zhijiang commented on FLINK-12889: -- My previous analysis in ML was as follows:   Actually all the five

[jira] [Updated] (FLINK-12889) Job keeps in FAILING state

2019-06-25 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12889: - Component/s: (was: API / DataStream) Runtime / Coordination > Job keeps in FAILING

[jira] [Commented] (FLINK-12909) add try catch when find a unique file name for the spilling channel

2019-06-24 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16871942#comment-16871942 ] zhijiang commented on FLINK-12909: -- Also you should connect the PR #7191 to this Jira ticket

[jira] [Commented] (FLINK-12909) add try catch when find a unique file name for the spilling channel

2019-06-24 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16871941#comment-16871941 ] zhijiang commented on FLINK-12909: -- The ticket does not describes the issue well. Any IOException 

[jira] [Created] (FLINK-12882) Remove ExecutionAttemptID field from ShuffleEnvironment#createResultPartitionWriters

2019-06-18 Thread zhijiang (JIRA)
zhijiang created FLINK-12882: Summary: Remove ExecutionAttemptID field from ShuffleEnvironment#createResultPartitionWriters Key: FLINK-12882 URL: https://issues.apache.org/jira/browse/FLINK-12882

[jira] [Updated] (FLINK-12882) Remove ExecutionAttemptID field from ShuffleEnvironment#createResultPartitionWriters

2019-06-18 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12882: - Description: The {{ExecutionAttemptID}} is only used for constructing {{ResultPartitionID}}  during

[jira] [Updated] (FLINK-12843) Refactor the pin logic in ReleaseOnConsumptionResultPartition

2019-06-17 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12843: - Description: The pin logic is for adding the reference counter based on number of subpartitions for

[jira] [Updated] (FLINK-12843) Refactor the pin logic in ReleaseOnConsumptionResultPartition

2019-06-17 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12843: - Summary: Refactor the pin logic in ReleaseOnConsumptionResultPartition (was: Refactor the pin logic in

[jira] [Comment Edited] (FLINK-12070) Make blocking result partitions consumable multiple times

2019-06-14 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863947#comment-16863947 ] zhijiang edited comment on FLINK-12070 at 6/14/19 11:09 AM: In Yingjie's

[jira] [Commented] (FLINK-12070) Make blocking result partitions consumable multiple times

2019-06-14 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863947#comment-16863947 ] zhijiang commented on FLINK-12070: -- In Yingjie's testing based on session mode, the container is

[jira] [Created] (FLINK-12843) Refactor the pin logic in ResultPartition

2019-06-13 Thread zhijiang (JIRA)
zhijiang created FLINK-12843: Summary: Refactor the pin logic in ResultPartition Key: FLINK-12843 URL: https://issues.apache.org/jira/browse/FLINK-12843 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-12842) Fix invalid check released state during ResultPartition#createSubpartitionView

2019-06-13 Thread zhijiang (JIRA)
zhijiang created FLINK-12842: Summary: Fix invalid check released state during ResultPartition#createSubpartitionView Key: FLINK-12842 URL: https://issues.apache.org/jira/browse/FLINK-12842 Project:

[jira] [Updated] (FLINK-12761) Fine grained resource management

2019-06-06 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12761: - Summary: Fine grained resource management (was: Fine grained resource managedment) > Fine grained

[jira] [Updated] (FLINK-12738) Remove abstract getPageSize method from InputGate

2019-06-05 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12738: - Summary: Remove abstract getPageSize method from InputGate (was: Remove abstract getPageSize from

[jira] [Commented] (FLINK-12740) SpillableSubpartitionTest deadlocks on Travis

2019-06-05 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856643#comment-16856643 ] zhijiang commented on FLINK-12740: -- Yes, the new added uint test for FLINK-12544 might be not stable.

[jira] [Assigned] (FLINK-12740) SpillableSubpartitionTest deadlocks on Travis

2019-06-05 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang reassigned FLINK-12740: Assignee: zhijiang > SpillableSubpartitionTest deadlocks on Travis >

[jira] [Created] (FLINK-12738) Remove abstract getPageSize from InputGate

2019-06-05 Thread zhijiang (JIRA)
zhijiang created FLINK-12738: Summary: Remove abstract getPageSize from InputGate Key: FLINK-12738 URL: https://issues.apache.org/jira/browse/FLINK-12738 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-12735) Make shuffle environment implementation independent with IOManager

2019-06-05 Thread zhijiang (JIRA)
zhijiang created FLINK-12735: Summary: Make shuffle environment implementation independent with IOManager Key: FLINK-12735 URL: https://issues.apache.org/jira/browse/FLINK-12735 Project: Flink

[jira] [Closed] (FLINK-11604) Extend the necessary methods in ResultPartitionWriter interface

2019-06-04 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang closed FLINK-11604. Resolution: Duplicate Already done in FLINK-12570 > Extend the necessary methods in

[jira] [Commented] (FLINK-12544) Deadlock while releasing memory and requesting segment concurrent in SpillableSubpartition

2019-06-04 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855696#comment-16855696 ] zhijiang commented on FLINK-12544: -- Merged into release-1.8 : 61fa1005adc61b6fc3439fdd127e5f25adea

[jira] [Closed] (FLINK-12544) Deadlock while releasing memory and requesting segment concurrent in SpillableSubpartition

2019-06-04 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang closed FLINK-12544. Resolution: Fixed Fix Version/s: 1.8.1 > Deadlock while releasing memory and requesting segment

[jira] [Commented] (FLINK-12201) Introduce InputGateWithMetrics in Task to increment numBytesIn metric

2019-06-04 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1684#comment-1684 ] zhijiang commented on FLINK-12201: -- master : 3351ca11a366713328a6d0cb0e87100cb8c24200 > Introduce

[jira] [Closed] (FLINK-12201) Introduce InputGateWithMetrics in Task to increment numBytesIn metric

2019-06-04 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang closed FLINK-12201. Resolution: Fixed > Introduce InputGateWithMetrics in Task to increment numBytesIn metric >

[jira] [Comment Edited] (FLINK-12070) Make blocking result partitions consumable multiple times

2019-06-03 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855252#comment-16855252 ] zhijiang edited comment on FLINK-12070 at 6/4/19 2:59 AM: -- Agree with Stephan

[jira] [Comment Edited] (FLINK-12070) Make blocking result partitions consumable multiple times

2019-06-03 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855252#comment-16855252 ] zhijiang edited comment on FLINK-12070 at 6/4/19 2:58 AM: -- Agree with Stephan

[jira] [Commented] (FLINK-12070) Make blocking result partitions consumable multiple times

2019-06-03 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855252#comment-16855252 ] zhijiang commented on FLINK-12070: -- Agree with Stephan and Stefan's opinions. We could add the batch

[jira] [Updated] (FLINK-12603) Remove getOwningTaskName method from InputGate

2019-06-03 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12603: - Summary: Remove getOwningTaskName method from InputGate (was: Refactor InputGate interface to remove

[jira] [Updated] (FLINK-12603) Refactor InputGate interface to remove unnecessary methods

2019-06-03 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12603: - Description: In order to make abstract InputGate simple for extending new implementations in shuffle

[jira] [Commented] (FLINK-12642) OutputBufferPoolUsageGauge can fail with NPE

2019-05-29 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16851462#comment-16851462 ] zhijiang commented on FLINK-12642: -- Yes, that is the ideal way which we confirmed before to make

[jira] [Updated] (FLINK-12544) Deadlock while releasing memory and requesting segment concurrent in SpillableSubpartition

2019-05-28 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12544: - Description: It is reported by flink user, and the original jstack is as following:   {code:java} //

[jira] [Updated] (FLINK-12544) Deadlock while releasing memory and requesting segment concurrent in SpillableSubpartition

2019-05-28 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12544: - Summary: Deadlock while releasing memory and requesting segment concurrent in SpillableSubpartition

[jira] [Commented] (FLINK-12642) OutputBufferPoolUsageGauge can fail with NPE

2019-05-28 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16849508#comment-16849508 ] zhijiang commented on FLINK-12642: -- Thanks for reporting this [~Zentol]! I think we could check

[jira] [Created] (FLINK-12630) Refactor abstract InputGate to general interface

2019-05-27 Thread zhijiang (JIRA)
zhijiang created FLINK-12630: Summary: Refactor abstract InputGate to general interface Key: FLINK-12630 URL: https://issues.apache.org/jira/browse/FLINK-12630 Project: Flink Issue Type:

[jira] [Updated] (FLINK-12201) Introduce InputGateWithMetrics in Task to increment numBytesIn metric

2019-05-24 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12201: - Description: Incrementing of numBytesIn metric in SingleInputGate does not depend on shuffle service

[jira] [Updated] (FLINK-12201) Introduce InputGateWithMetrics in Task to increment numBytesIn metric

2019-05-24 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12201: - Summary: Introduce InputGateWithMetrics in Task to increment numBytesIn metric (was: Introduce

[jira] [Updated] (FLINK-12603) Refactor InputGate interface to remove unnecessary methods

2019-05-24 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12603: - Description: In order to make abstract InputGate simple for extending new implementations in shuffle

[jira] [Updated] (FLINK-12603) Refactor InputGate interface to remove unnecessary methods

2019-05-24 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12603: - Summary: Refactor InputGate interface to remove unnecessary methods (was: Remove getOwningTaskName

[jira] [Created] (FLINK-12603) Remove getOwningTaskName method from InputGate

2019-05-23 Thread zhijiang (JIRA)
zhijiang created FLINK-12603: Summary: Remove getOwningTaskName method from InputGate Key: FLINK-12603 URL: https://issues.apache.org/jira/browse/FLINK-12603 Project: Flink Issue Type: Sub-task

[jira] [Updated] (FLINK-6227) Introduce the abstract PartitionException for downstream task failure

2019-05-22 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-6227: Summary: Introduce the abstract PartitionException for downstream task failure (was: Introduce the

[jira] [Updated] (FLINK-6227) Introduce the DataConsumptionException for downstream task failure

2019-05-22 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-6227: Description: It is part of FLIP-1.   We define a new abstract special exception called

[jira] [Updated] (FLINK-12458) Introduce PartitionConnectionException for unreachable producer

2019-05-22 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12458: - Description: If the consumer can not establish a connection to remote task executor, which might

[jira] [Updated] (FLINK-12458) Introduce PartitionConnectionException for unreachable producer

2019-05-22 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12458: - Summary: Introduce PartitionConnectionException for unreachable producer (was: Throw

[jira] [Created] (FLINK-12571) Make NetworkEnvironment#start() return the binded data port

2019-05-21 Thread zhijiang (JIRA)
zhijiang created FLINK-12571: Summary: Make NetworkEnvironment#start() return the binded data port Key: FLINK-12571 URL: https://issues.apache.org/jira/browse/FLINK-12571 Project: Flink Issue

[jira] [Commented] (FLINK-10653) Introduce Pluggable Shuffle Manager Architecture

2019-05-21 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-10653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844642#comment-16844642 ] zhijiang commented on FLINK-10653: -- Thanks for the concerns. [~prudenko] Actually we ever also

[jira] [Created] (FLINK-12564) Remove getBufferProvider method from ResultPartitionWriter interface

2019-05-20 Thread zhijiang (JIRA)
zhijiang created FLINK-12564: Summary: Remove getBufferProvider method from ResultPartitionWriter interface Key: FLINK-12564 URL: https://issues.apache.org/jira/browse/FLINK-12564 Project: Flink

[jira] [Created] (FLINK-12544) Deadlock during releasing memory in SpillableSubpartition

2019-05-17 Thread zhijiang (JIRA)
zhijiang created FLINK-12544: Summary: Deadlock during releasing memory in SpillableSubpartition Key: FLINK-12544 URL: https://issues.apache.org/jira/browse/FLINK-12544 Project: Flink Issue

[jira] [Commented] (FLINK-12202) Consider introducing batch metric register in NetworkEnviroment

2019-05-14 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839963#comment-16839963 ] zhijiang commented on FLINK-12202: -- Hey [~azagrebin], as we confirmed before, this ticket is not needed

[jira] [Closed] (FLINK-12474) UnionInputGate should be notified when closing SingleInputGate by canceler thread

2019-05-14 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang closed FLINK-12474. Resolution: Invalid > UnionInputGate should be notified when closing SingleInputGate by canceler >

[jira] [Commented] (FLINK-12474) UnionInputGate should be notified when closing SingleInputGate by canceler thread

2019-05-14 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839298#comment-16839298 ] zhijiang commented on FLINK-12474: -- Yes, you are right after double checking the codes. Because the

[jira] [Commented] (FLINK-12474) UnionInputGate should be notified when closing SingleInputGate by canceler thread

2019-05-14 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839230#comment-16839230 ] zhijiang commented on FLINK-12474: -- [~pnowojski] thanks for the confirmation! Yes, the

[jira] [Updated] (FLINK-12497) Refactor the start method of ConnectionManager

2019-05-13 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-12497: - Description: In current {{ConnectionManager#start(ResultPartitionProvider, TaskEventDispatcher)}}, the

[jira] [Created] (FLINK-12497) Refactor the start method of ConnectionManager

2019-05-13 Thread zhijiang (JIRA)
zhijiang created FLINK-12497: Summary: Refactor the start method of ConnectionManager Key: FLINK-12497 URL: https://issues.apache.org/jira/browse/FLINK-12497 Project: Flink Issue Type: Sub-task

[jira] [Updated] (FLINK-11403) Introduce ResultPartitionWithConsumableNotifier in task for notifying consumable result partition

2019-05-10 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-11403: - Summary: Introduce ResultPartitionWithConsumableNotifier in task for notifying consumable result

[jira] [Updated] (FLINK-11403) Remove ResultPartitionConsumableNotifier from ResultPartition

2019-05-10 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-11403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-11403: - Description: The creation of {{ResultPartitionWriter}} from {{NetworkEnvironment}} relies on

[jira] [Commented] (FLINK-12474) UnionInputGate should be notified when closing SingleInputGate by canceler thread

2019-05-10 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16836964#comment-16836964 ] zhijiang commented on FLINK-12474: -- [~pnowojski], do you think this jira is worth fixing or my

[jira] [Created] (FLINK-12474) UnionInputGate should be notified when closing SingleInputGate by canceler thread

2019-05-10 Thread zhijiang (JIRA)
zhijiang created FLINK-12474: Summary: UnionInputGate should be notified when closing SingleInputGate by canceler thread Key: FLINK-12474 URL: https://issues.apache.org/jira/browse/FLINK-12474 Project:

[jira] [Created] (FLINK-12458) Throw PartitionNotFoundException if consumer can not establish a connection to remote TM

2019-05-09 Thread zhijiang (JIRA)
zhijiang created FLINK-12458: Summary: Throw PartitionNotFoundException if consumer can not establish a connection to remote TM Key: FLINK-12458 URL: https://issues.apache.org/jira/browse/FLINK-12458

[jira] [Created] (FLINK-12331) Introduce partition/gate setup to decouple task registration with NetworkEnvironment

2019-04-25 Thread zhijiang (JIRA)
zhijiang created FLINK-12331: Summary: Introduce partition/gate setup to decouple task registration with NetworkEnvironment Key: FLINK-12331 URL: https://issues.apache.org/jira/browse/FLINK-12331

[jira] [Updated] (FLINK-6227) Introduce the DataConsumptionException for downstream task failure

2019-04-22 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang updated FLINK-6227: Component/s: Runtime / Coordination > Introduce the DataConsumptionException for downstream task failure >

[jira] [Created] (FLINK-12213) Pass TaskManagerMetricGroup into constructor of NetworkEnvironment

2019-04-16 Thread zhijiang (JIRA)
zhijiang created FLINK-12213: Summary: Pass TaskManagerMetricGroup into constructor of NetworkEnvironment Key: FLINK-12213 URL: https://issues.apache.org/jira/browse/FLINK-12213 Project: Flink

[jira] [Comment Edited] (FLINK-12070) Make blocking result partitions consumable multiple times

2019-04-15 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16818619#comment-16818619 ] zhijiang edited comment on FLINK-12070 at 4/16/19 4:25 AM: --- Thanks for

[jira] [Comment Edited] (FLINK-12070) Make blocking result partitions consumable multiple times

2019-04-15 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16818619#comment-16818619 ] zhijiang edited comment on FLINK-12070 at 4/16/19 4:24 AM: --- Thanks for

[jira] [Commented] (FLINK-12070) Make blocking result partitions consumable multiple times

2019-04-15 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16818619#comment-16818619 ] zhijiang commented on FLINK-12070: -- Thanks for providing the POC for this feature [~StephanEwen] I

[jira] [Commented] (FLINK-6227) Introduce the DataConsumptionException for downstream task failure

2019-04-15 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16818018#comment-16818018 ] zhijiang commented on FLINK-6227: - Ok, we would focus on the implementation soon. Also we could fully

[jira] [Created] (FLINK-12154) Remove legacy fields for SingleInputGate

2019-04-10 Thread zhijiang (JIRA)
zhijiang created FLINK-12154: Summary: Remove legacy fields for SingleInputGate Key: FLINK-12154 URL: https://issues.apache.org/jira/browse/FLINK-12154 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-12146) Remove unregister task from NetworkEnvironment to simplify the interface of ShuffleService

2019-04-09 Thread zhijiang (JIRA)
zhijiang created FLINK-12146: Summary: Remove unregister task from NetworkEnvironment to simplify the interface of ShuffleService Key: FLINK-12146 URL: https://issues.apache.org/jira/browse/FLINK-12146

[jira] [Commented] (FLINK-6227) Introduce the DataConsumptionException for downstream task failure

2019-04-09 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813215#comment-16813215 ] zhijiang commented on FLINK-6227: - Exactly, this exception is received on consumer side via network stack

[jira] [Created] (FLINK-12127) Move network related options to NetworkEnvironmentOptions

2019-04-08 Thread zhijiang (JIRA)
zhijiang created FLINK-12127: Summary: Move network related options to NetworkEnvironmentOptions Key: FLINK-12127 URL: https://issues.apache.org/jira/browse/FLINK-12127 Project: Flink Issue

[jira] [Assigned] (FLINK-6227) Introduce the DataConsumptionException for downstream task failure

2019-04-02 Thread zhijiang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhijiang reassigned FLINK-6227: --- Assignee: zhijiang > Introduce the DataConsumptionException for downstream task failure >

<    4   5   6   7   8   9   10   11   12   >