[jira] [Created] (FLINK-5830) OutOfMemoryError during notify final state in TaskExecutor may cause job stuck

2017-02-17 Thread Zhijiang Wang (JIRA)
Zhijiang Wang created FLINK-5830: Summary: OutOfMemoryError during notify final state in TaskExecutor may cause job stuck Key: FLINK-5830 URL: https://issues.apache.org/jira/browse/FLINK-5830

[jira] [Commented] (FLINK-5799) Let RpcService.scheduleRunnable return ScheduledFuture

2017-02-14 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867126#comment-15867126 ] Zhijiang Wang commented on FLINK-5799: -- This fix is useful and the {{HeartbeatManagerSenderImpl}} can

[jira] [Comment Edited] (FLINK-5799) Let RpcService.scheduleRunnable return ScheduledFuture

2017-02-14 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867126#comment-15867126 ] Zhijiang Wang edited comment on FLINK-5799 at 2/15/17 2:56 AM: --- This fix is

[jira] [Commented] (FLINK-5798) Let the RPCService provide a ScheduledExecutorService

2017-02-14 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867116#comment-15867116 ] Zhijiang Wang commented on FLINK-5798: -- That is good, and the heartbeat manager component can reuse

[jira] [Updated] (FLINK-5133) Support to set resource for operator in DataStream and DataSet

2017-02-14 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5133: - Summary: Support to set resource for operator in DataStream and DataSet (was: Add new

[jira] [Assigned] (FLINK-5501) Determine whether the job starts from last JobManager failure

2017-02-13 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang reassigned FLINK-5501: Assignee: shuai.xu (was: Zhijiang Wang) > Determine whether the job starts from last

[jira] [Assigned] (FLINK-5703) ExecutionGraph recovery based on reconciliation with TaskManager reports

2017-02-13 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang reassigned FLINK-5703: Assignee: Zhijiang Wang (was: shuai.xu) > ExecutionGraph recovery based on reconciliation

[jira] [Assigned] (FLINK-5703) ExecutionGraph recovery based on reconciliation with TaskManager reports

2017-02-13 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang reassigned FLINK-5703: Assignee: (was: Zhijiang Wang) > ExecutionGraph recovery based on reconciliation with

[jira] [Assigned] (FLINK-5703) ExecutionGraph recovery based on reconciliation with TaskManager reports

2017-02-13 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang reassigned FLINK-5703: Assignee: shuai.xu > ExecutionGraph recovery based on reconciliation with TaskManager

[jira] [Commented] (FLINK-5132) Introduce the ResourceSpec for grouping different resource factors in API

2017-02-09 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860543#comment-15860543 ] Zhijiang Wang commented on FLINK-5132: -- Thank you, [~StephanEwen]. Yeah, it also makes sense to check

[jira] [Updated] (FLINK-5703) ExecutionGraph recovery based on reconciliation with TaskManager reports

2017-02-03 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5703: - Description: The ExecutionGraph structure would be recovered from TaskManager reports during

[jira] [Updated] (FLINK-5703) ExecutionGraph recovery based on reconciliation with TaskManager reports

2017-02-03 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5703: - Description: The ExecutionGraph structure would be recovered from TaskManager reports during

[jira] [Updated] (FLINK-5703) ExecutionGraph recovery based on reconciliation with TaskManager reports

2017-02-03 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5703: - Description: The ExecutionGraph structure would be recovered from TaskManager reports during

[jira] [Created] (FLINK-5703) ExecutionGraph recovery based on reconciliation with TaskManager reports

2017-02-02 Thread Zhijiang Wang (JIRA)
Zhijiang Wang created FLINK-5703: Summary: ExecutionGraph recovery based on reconciliation with TaskManager reports Key: FLINK-5703 URL: https://issues.apache.org/jira/browse/FLINK-5703 Project:

[jira] [Commented] (FLINK-4364) Implement TaskManager side of heartbeat from JobManager

2017-01-18 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827892#comment-15827892 ] Zhijiang Wang commented on FLINK-4364: -- [~till.rohrmann], I closed the previous pull request for this

[jira] [Comment Edited] (FLINK-5501) Determine whether the job starts from last JobManager failure

2017-01-18 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827361#comment-15827361 ] Zhijiang Wang edited comment on FLINK-5501 at 1/18/17 8:42 AM: ---

[jira] [Comment Edited] (FLINK-5501) Determine whether the job starts from last JobManager failure

2017-01-17 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827361#comment-15827361 ] Zhijiang Wang edited comment on FLINK-5501 at 1/18/17 3:16 AM: --- Thank you

[jira] [Commented] (FLINK-5501) Determine whether the job starts from last JobManager failure

2017-01-17 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827361#comment-15827361 ] Zhijiang Wang commented on FLINK-5501: -- Thank you for the quick response! Yeah, you already

[jira] [Updated] (FLINK-5501) Determine whether the job starts from last JobManager failure

2017-01-16 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5501: - Description: When the {{JobManagerRunner}} grants leadership, it should check whether the

[jira] [Updated] (FLINK-5501) Determine whether the job starts from last JobManager failure

2017-01-16 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5501: - Description: When the {{JobManagerRunner}} grants leadership, it should check whether the

[jira] [Assigned] (FLINK-5501) Determine whether the job starts from last JobManager failure

2017-01-16 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang reassigned FLINK-5501: Assignee: Zhijiang Wang > Determine whether the job starts from last JobManager failure >

[jira] [Created] (FLINK-5501) Determine whether the job starts from last JobManager failure

2017-01-16 Thread Zhijiang Wang (JIRA)
Zhijiang Wang created FLINK-5501: Summary: Determine whether the job starts from last JobManager failure Key: FLINK-5501 URL: https://issues.apache.org/jira/browse/FLINK-5501 Project: Flink

[jira] [Assigned] (FLINK-5499) Try to reuse the resource location of prior execution attempt in allocating slot

2017-01-15 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang reassigned FLINK-5499: Assignee: Zhijiang Wang > Try to reuse the resource location of prior execution attempt in

[jira] [Updated] (FLINK-5499) Try to reuse the resource location of prior execution attempt in allocating slot

2017-01-15 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5499: - Description: Currently when schedule execution to request to allocate slot from {{SlotPool}}, the

[jira] [Updated] (FLINK-5499) Try to reuse the resource location of prior execution attempt in allocating slot

2017-01-15 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5499: - Description: Currently when schedule execution to request to allocate slot from {{SlotPool}}, the

[jira] [Updated] (FLINK-5499) Try to reuse the resource location of prior execution attempt in allocating slot

2017-01-15 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5499: - Summary: Try to reuse the resource location of prior execution attempt in allocating slot (was:

[jira] [Updated] (FLINK-5499) Try to reuse the resource location of prior execution attempt when allocate slot

2017-01-15 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5499: - Summary: Try to reuse the resource location of prior execution attempt when allocate slot (was:

[jira] [Created] (FLINK-5499) Try to reuse the resource location of previous execution attempt when allocate slot

2017-01-15 Thread Zhijiang Wang (JIRA)
Zhijiang Wang created FLINK-5499: Summary: Try to reuse the resource location of previous execution attempt when allocate slot Key: FLINK-5499 URL: https://issues.apache.org/jira/browse/FLINK-5499

[jira] [Assigned] (FLINK-5138) The StateBackend provides the method to estimate memory usage based on hint of state size in ResourceSpec

2016-11-30 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang reassigned FLINK-5138: Assignee: Zhijiang Wang > The StateBackend provides the method to estimate memory usage

[jira] [Updated] (FLINK-5138) The StateBackend provides the method to estimate memory usage based on hint of state size in ResourceSpec

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5138: - Description: Users may specify the state size in *setResource* API, then the different

[jira] [Updated] (FLINK-5138) The StateBackend provides the method to estimate memory usage based on hint of state size in ResourceSpec

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5138: - Component/s: Core > The StateBackend provides the method to estimate memory usage based on hint

[jira] [Created] (FLINK-5138) The StateBackend provides the method to estimate memory usage based on hint of state size in ResourceSpec

2016-11-22 Thread Zhijiang Wang (JIRA)
Zhijiang Wang created FLINK-5138: Summary: The StateBackend provides the method to estimate memory usage based on hint of state size in ResourceSpec Key: FLINK-5138 URL:

[jira] [Updated] (FLINK-5131) Fine-grained Resource Configuration

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5131: - Description: Normally the UDF just creates short-life small objects and these can be recycled

[jira] [Assigned] (FLINK-5134) Aggregate ResourceSpe for chained operators when generating job graph

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang reassigned FLINK-5134: Assignee: Zhijiang Wang > Aggregate ResourceSpe for chained operators when generating job

[jira] [Assigned] (FLINK-5133) Add new setResource API for DataStream and DataSet

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang reassigned FLINK-5133: Assignee: Zhijiang Wang > Add new setResource API for DataStream and DataSet >

[jira] [Assigned] (FLINK-5136) Aggregation of slot ResourceProfile and framework memory by ResourceManager

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang reassigned FLINK-5136: Assignee: Zhijiang Wang > Aggregation of slot ResourceProfile and framework memory by

[jira] [Assigned] (FLINK-5135) ResourceProfile for slot request should be expanded to correspond with ResourceSpec

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang reassigned FLINK-5135: Assignee: Zhijiang Wang > ResourceProfile for slot request should be expanded to

[jira] [Assigned] (FLINK-5132) Introduce the ResourceSpec for grouping different resource factors in API

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang reassigned FLINK-5132: Assignee: Zhijiang Wang > Introduce the ResourceSpec for grouping different resource

[jira] [Assigned] (FLINK-5137) Config JVM options and set onto the TaskManager container

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang reassigned FLINK-5137: Assignee: Zhijiang Wang > Config JVM options and set onto the TaskManager container >

[jira] [Updated] (FLINK-5137) Config JVM options and set onto the TaskManager container

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5137: - Description: The users can config JVM options in job level. And the *ResourceManager* will set

[jira] [Created] (FLINK-5137) Config JVM options and set onto the TaskManager container

2016-11-22 Thread Zhijiang Wang (JIRA)
Zhijiang Wang created FLINK-5137: Summary: Config JVM options and set onto the TaskManager container Key: FLINK-5137 URL: https://issues.apache.org/jira/browse/FLINK-5137 Project: Flink

[jira] [Updated] (FLINK-5136) Aggregation of slot ResourceProfile and framework memory by ResourceManager

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5136: - Description: When *ResourceManager* requests container resource from cluster, the framework

[jira] [Updated] (FLINK-5136) Aggregation of slot ResourceProfile and framework memory by ResourceManager

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5136: - Summary: Aggregation of slot ResourceProfile and framework memory by ResourceManager (was:

[jira] [Updated] (FLINK-5136) Aggregation of slot ResourceProfile and framework memory for requesting resource from cluster by ResourceManager

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5136: - Summary: Aggregation of slot ResourceProfile and framework memory for requesting resource from

[jira] [Created] (FLINK-5136) ResourceManager should consider both slot resource profile and framework memory for requesting resource from cluster

2016-11-22 Thread Zhijiang Wang (JIRA)
Zhijiang Wang created FLINK-5136: Summary: ResourceManager should consider both slot resource profile and framework memory for requesting resource from cluster Key: FLINK-5136 URL:

[jira] [Updated] (FLINK-5135) ResourceProfile for slot request should be expanded to correspond with ResourceSpec

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5135: - Description: The *JobManager* requests slot by *ResourceProfile* from *ResourceManager* before

[jira] [Updated] (FLINK-5135) ResourceProfile for slot request should be expanded to correspond with ResourceSpec

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5135: - Summary: ResourceProfile for slot request should be expanded to correspond with ResourceSpec

[jira] [Updated] (FLINK-5131) Fine-grained Resource Configuration

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5131: - Description: Normally the UDF just creates short-life small objects and these can be recycled

[jira] [Created] (FLINK-5135) ResourceProfile should be expanded to correspond with ResourceSpec

2016-11-22 Thread Zhijiang Wang (JIRA)
Zhijiang Wang created FLINK-5135: Summary: ResourceProfile should be expanded to correspond with ResourceSpec Key: FLINK-5135 URL: https://issues.apache.org/jira/browse/FLINK-5135 Project: Flink

[jira] [Updated] (FLINK-5134) Aggregate ResourceSpe for chained operators when generating job graph

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5134: - Description: In *JobGraph* generation, each *JobVertex* corresponds to a series of chained

[jira] [Updated] (FLINK-5134) Aggregate ResourceSpe for chained operators when generating job graph

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5134: - Description: In *JobGraph* generation, each *JobVertex* corresponds to a series of chained

[jira] [Updated] (FLINK-5131) Fine-grained Resource Configuration

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5131: - Description: Normally the UDF just creates short-life small objects and these can be recycled

[jira] [Updated] (FLINK-5134) Aggregate ResourceSpe for chained operators when generating job graph

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5134: - Summary: Aggregate ResourceSpe for chained operators when generating job graph (was: Aggregate

[jira] [Updated] (FLINK-5134) Aggregate ResourceSpe for chained operators when generating stream graph

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5134: - Description: In DataStream API, the *ResourceSpec* is setted onto the internal transformation

[jira] [Updated] (FLINK-5134) Aggregate ResourceSpe for chained operators when generating stream graph

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5134: - Description: In datastream API, > Aggregate ResourceSpe for chained operators when generating

[jira] [Created] (FLINK-5134) Aggregate ResourceSpe for chained operators when generating stream graph

2016-11-22 Thread Zhijiang Wang (JIRA)
Zhijiang Wang created FLINK-5134: Summary: Aggregate ResourceSpe for chained operators when generating stream graph Key: FLINK-5134 URL: https://issues.apache.org/jira/browse/FLINK-5134 Project:

[jira] [Updated] (FLINK-5133) Add new setResource API for DataStream and DataSet

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5133: - Description: This is part of the fine-grained resource configuration. For *DataStream*, the

[jira] [Created] (FLINK-5133) Add new setResource API for DataStream and DataSet

2016-11-22 Thread Zhijiang Wang (JIRA)
Zhijiang Wang created FLINK-5133: Summary: Add new setResource API for DataStream and DataSet Key: FLINK-5133 URL: https://issues.apache.org/jira/browse/FLINK-5133 Project: Flink Issue Type:

[jira] [Updated] (FLINK-5131) Fine-grained Resource Configuration

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5131: - Description: Normally the UDF just creates short-life small objects and these can be recycled

[jira] [Updated] (FLINK-5132) Introduce the ResourceSpec for grouping different resource factors in API

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5132: - Description: This is part of the fine-grained resource configuration. The current resource

[jira] [Updated] (FLINK-5132) Introduce the ResourceSpec for grouping different resource factors in API

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5132: - Description: This is part of the fine-grained resource configuration. The current resource

[jira] [Created] (FLINK-5132) Introduce the ResourceSpec for grouping different resource factors in API

2016-11-22 Thread Zhijiang Wang (JIRA)
Zhijiang Wang created FLINK-5132: Summary: Introduce the ResourceSpec for grouping different resource factors in API Key: FLINK-5132 URL: https://issues.apache.org/jira/browse/FLINK-5132 Project:

[jira] [Updated] (FLINK-5131) Fine-grained Resource Configuration

2016-11-22 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-5131: - Description: Normally the UDF just creates short-life small objects and these can be recycled

[jira] [Created] (FLINK-5131) Fine-grained Resource Configuration

2016-11-22 Thread Zhijiang Wang (JIRA)
Zhijiang Wang created FLINK-5131: Summary: Fine-grained Resource Configuration Key: FLINK-5131 URL: https://issues.apache.org/jira/browse/FLINK-5131 Project: Flink Issue Type: New Feature

[jira] [Commented] (FLINK-4364) Implement TaskManager side of heartbeat from JobManager

2016-11-09 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15650576#comment-15650576 ] Zhijiang Wang commented on FLINK-4364: -- Hi [~till.rohrmann], for the heartbeat interaction between TM

[jira] [Commented] (FLINK-4354) Implement TaskManager side of heartbeat from ResourceManager

2016-11-07 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15646297#comment-15646297 ] Zhijiang Wang commented on FLINK-4354: -- Actually the current {{HeartbeatManagerImpl}} can work and be

[jira] [Comment Edited] (FLINK-4354) Implement TaskManager side of heartbeat from ResourceManager

2016-11-07 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15643637#comment-15643637 ] Zhijiang Wang edited comment on FLINK-4354 at 11/7/16 9:46 AM: --- Hi

[jira] [Comment Edited] (FLINK-4354) Implement TaskManager side of heartbeat from ResourceManager

2016-11-07 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15643637#comment-15643637 ] Zhijiang Wang edited comment on FLINK-4354 at 11/7/16 9:44 AM: --- Hi

[jira] [Comment Edited] (FLINK-4354) Implement TaskManager side of heartbeat from ResourceManager

2016-11-07 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15643637#comment-15643637 ] Zhijiang Wang edited comment on FLINK-4354 at 11/7/16 9:41 AM: --- Hi

[jira] [Commented] (FLINK-4354) Implement TaskManager side of heartbeat from ResourceManager

2016-11-07 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15643637#comment-15643637 ] Zhijiang Wang commented on FLINK-4354: -- Hi [~till.rohrmann], for heartbeat interaction between TM and

[jira] [Updated] (FLINK-4911) Non-disruptive JobManager Failures via Reconciliation

2016-10-27 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-4911: - Component/s: TaskManager > Non-disruptive JobManager Failures via Reconciliation >

[jira] [Assigned] (FLINK-4911) Non-disruptive JobManager Failures via Reconciliation

2016-10-26 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang reassigned FLINK-4911: Assignee: Zhijiang Wang > Non-disruptive JobManager Failures via Reconciliation >

[jira] [Commented] (FLINK-4911) Non-disruptive JobManager Failures via Reconciliation

2016-10-26 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15607630#comment-15607630 ] Zhijiang Wang commented on FLINK-4911: -- Hi Stephan, the sub-tasks 2 and 3 are the same, do you mind I

[jira] [Commented] (FLINK-4911) Non-disruptive JobManager Failures via Reconciliation

2016-10-25 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15607336#comment-15607336 ] Zhijiang Wang commented on FLINK-4911: -- FLINK-3539 tries to realize eventual consistency between JM

[jira] [Assigned] (FLINK-4912) Introduce RECONCILING state in ExecutionGraph

2016-10-25 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang reassigned FLINK-4912: Assignee: Zhijiang Wang > Introduce RECONCILING state in ExecutionGraph >

[jira] [Assigned] (FLINK-4915) Define parameters for reconciliation

2016-10-25 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang reassigned FLINK-4915: Assignee: Zhijiang Wang > Define parameters for reconciliation >

[jira] [Assigned] (FLINK-4914) Define parameters for reconciliation

2016-10-25 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang reassigned FLINK-4914: Assignee: Zhijiang Wang > Define parameters for reconciliation >

[jira] [Comment Edited] (FLINK-4715) TaskManager should commit suicide after cancellation failure

2016-09-29 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15534797#comment-15534797 ] Zhijiang Wang edited comment on FLINK-4715 at 9/30/16 2:31 AM: --- Yes, we

[jira] [Commented] (FLINK-4715) TaskManager should commit suicide after cancellation failure

2016-09-29 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15534797#comment-15534797 ] Zhijiang Wang commented on FLINK-4715: -- Yes, we already experienced this problem in real production

[jira] [Closed] (FLINK-4505) Implement TaskManagerRunner to construct related components for TaskManager

2016-09-28 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang closed FLINK-4505. > Implement TaskManagerRunner to construct related components for TaskManager >

[jira] [Updated] (FLINK-4505) Implement TaskManagerRunner to construct related components for TaskManager

2016-09-09 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-4505: - Description: Implement {{TaskManagerRunner}} to construct related components ({{MemoryManager}},

[jira] [Updated] (FLINK-4505) Implement TaskManagerRunner to construct related components for TaskManager

2016-09-09 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-4505: - Summary: Implement TaskManagerRunner to construct related components for TaskManager (was:

[jira] [Commented] (FLINK-4363) Implement TaskManager basic startup of all components in java

2016-08-26 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15438745#comment-15438745 ] Zhijiang Wang commented on FLINK-4363: -- Hi [~mxm], I have created the FLINK-4505 for

[jira] [Updated] (FLINK-4505) Implement TaskManagerFactory to bring up TaskManager for different modes

2016-08-26 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-4505: - Summary: Implement TaskManagerFactory to bring up TaskManager for different modes (was:

[jira] [Assigned] (FLINK-4505) Implement TaskManagerFactory to bring up TaskManager for different modes

2016-08-26 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang reassigned FLINK-4505: Assignee: Zhijiang Wang > Implement TaskManagerFactory to bring up TaskManager for

[jira] [Created] (FLINK-4505) Implement abstract TaskManagerFactory to bring up TaskManager for different modes

2016-08-26 Thread Zhijiang Wang (JIRA)
Zhijiang Wang created FLINK-4505: Summary: Implement abstract TaskManagerFactory to bring up TaskManager for different modes Key: FLINK-4505 URL: https://issues.apache.org/jira/browse/FLINK-4505

[jira] [Commented] (FLINK-4489) Implement TaskManager's SlotManager

2016-08-26 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15438621#comment-15438621 ] Zhijiang Wang commented on FLINK-4489: -- Does it refer to the slot manager in {{ResourceManager}} or

[jira] [Commented] (FLINK-4354) Implement TaskManager side of heartbeat from ResourceManager

2016-08-26 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15438609#comment-15438609 ] Zhijiang Wang commented on FLINK-4354: -- Yes, so it would be started after flink-4478 done. >

[jira] [Commented] (FLINK-4424) Make network environment start-up/shutdown independent of JobManager association

2016-08-24 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15434680#comment-15434680 ] Zhijiang Wang commented on FLINK-4424: -- The previous way of {{NetworkEnvironment}} is like lazy

[jira] [Commented] (FLINK-4363) Implement TaskManager basic startup of all components in java

2016-08-19 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15427951#comment-15427951 ] Zhijiang Wang commented on FLINK-4363: -- Thank you Till, I got your meaning. I will follow the

[jira] [Assigned] (FLINK-4363) Implement TaskManager basic startup of all components in java

2016-08-17 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang reassigned FLINK-4363: Assignee: Zhijiang Wang > Implement TaskManager basic startup of all components in java >

[jira] [Commented] (FLINK-4021) Problem of setting autoread for netty channel when more tasks sharing the same Tcp connection

2016-08-17 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15424183#comment-15424183 ] Zhijiang Wang commented on FLINK-4021: -- got it, i will modify the return and add the test to PR

[jira] [Commented] (FLINK-4363) Implement TaskManager basic startup of all components in java

2016-08-17 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15424056#comment-15424056 ] Zhijiang Wang commented on FLINK-4363: -- For yarn and standalone mode, the entrance methods for

[jira] [Assigned] (FLINK-4355) Implement TaskManager side of registration at ResourceManager

2016-08-17 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang reassigned FLINK-4355: Assignee: Stephan Ewen (was: Zhijiang Wang) > Implement TaskManager side of registration

[jira] [Commented] (FLINK-4021) Problem of setting autoread for netty channel when more tasks sharing the same Tcp connection

2016-08-17 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15423994#comment-15423994 ] Zhijiang Wang commented on FLINK-4021: -- Yeah, you are right. It is no problem in current flink

[jira] [Commented] (FLINK-4021) Problem of setting autoread for netty channel when more tasks sharing the same Tcp connection

2016-08-17 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15423984#comment-15423984 ] Zhijiang Wang commented on FLINK-4021: -- Thank you for advice, it is indeed simple by 'return

[jira] [Updated] (FLINK-4364) Implement TaskManager side of heartbeat from JobManager

2016-08-11 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-4364: - Description: The {{JobManager}} initiates heartbeat messages via (JobID, JmLeaderID), and the

[jira] [Updated] (FLINK-4357) Implement TaskManager side of slot allocation from ResourceManager

2016-08-11 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-4357: - Description: The {{ResourceManager}} may tell the {{TaskManager}} to give a slot to a specific

[jira] [Updated] (FLINK-4357) Implement TaskManager side of slot allocation from ResourceManager

2016-08-11 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang updated FLINK-4357: - Description: The {{ResourceManager}} may tell the {{TaskManager}} to give a slot to a specific

[jira] [Assigned] (FLINK-4357) Implement TaskManager side of slot allocation from ResourceManager

2016-08-10 Thread Zhijiang Wang (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijiang Wang reassigned FLINK-4357: Assignee: Zhijiang Wang > Implement TaskManager side of slot allocation from

  1   2   >