[jira] [Commented] (FLINK-6521) Add per job cleanup methods to HighAvailabilityServices

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-6521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108362#comment-16108362
 ] 

ASF GitHub Bot commented on FLINK-6521:
---

Github user zjureel commented on the issue:

https://github.com/apache/flink/pull/4376
  
I found the following kinda stuff from CI, and it seems not relevant to 
this issue, what do you think? @tillrohrmann 
```
Running org.apache.flink.test.recovery.JobManagerHACheckpointRecoveryITCase
java.lang.RuntimeException: org.apache.zookeeper.server.ZooKeeperServer 
class is frozen
at javassist.CtClassType.checkModify(CtClassType.java:288)
at javassist.CtBehavior.setBody(CtBehavior.java:432)
at javassist.CtBehavior.setBody(CtBehavior.java:412)
at 
org.apache.curator.test.ByteCodeRewrite.fixMethods(ByteCodeRewrite.java:91)
at 
org.apache.curator.test.ByteCodeRewrite.(ByteCodeRewrite.java:50)
at org.apache.curator.test.TestingServer.(TestingServer.java:33)
at 
org.apache.flink.test.recovery.JobManagerHACheckpointRecoveryITCase.testCheckpointedStreamingProgram(JobManagerHACheckpointRecoveryITCase.java:350)
at 
org.apache.flink.test.recovery.JobManagerHACheckpointRecoveryITCase.testCheckpointedStreamingProgramIncrementalRocksDB(JobManagerHACheckpointRecoveryITCase.java:336)
```


> Add per job cleanup methods to HighAvailabilityServices
> ---
>
> Key: FLINK-6521
> URL: https://issues.apache.org/jira/browse/FLINK-6521
> Project: Flink
>  Issue Type: Improvement
>  Components: Distributed Coordination
>Affects Versions: 1.3.0, 1.4.0
>Reporter: Till Rohrmann
>Assignee: Fang Yong
>
> The {{HighAvailabilityServices}} are used to manage services and persistent 
> state at a single point. This also entails the cleanup of data used for HA. 
> So far the {{HighAvailabilityServices}} can only clean up the data for all 
> stored jobs. In order to support cluster sessions, we have to extend this 
> functionality to selectively delete data for single jobs. This is necessary 
> to keep data for failed jobs and delete data for successfully executed jobs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink issue #4376: [FLINK-6521] Add per job cleanup methods to HighAvailabil...

[zjureel]

Github user zjureel commented on the issue:

https://github.com/apache/flink/pull/4376
  
I found the following kinda stuff from CI, and it seems not relevant to 
this issue, what do you think? @tillrohrmann 
```
Running org.apache.flink.test.recovery.JobManagerHACheckpointRecoveryITCase
java.lang.RuntimeException: org.apache.zookeeper.server.ZooKeeperServer 
class is frozen
at javassist.CtClassType.checkModify(CtClassType.java:288)
at javassist.CtBehavior.setBody(CtBehavior.java:432)
at javassist.CtBehavior.setBody(CtBehavior.java:412)
at 
org.apache.curator.test.ByteCodeRewrite.fixMethods(ByteCodeRewrite.java:91)
at 
org.apache.curator.test.ByteCodeRewrite.(ByteCodeRewrite.java:50)
at org.apache.curator.test.TestingServer.(TestingServer.java:33)
at 
org.apache.flink.test.recovery.JobManagerHACheckpointRecoveryITCase.testCheckpointedStreamingProgram(JobManagerHACheckpointRecoveryITCase.java:350)
at 
org.apache.flink.test.recovery.JobManagerHACheckpointRecoveryITCase.testCheckpointedStreamingProgramIncrementalRocksDB(JobManagerHACheckpointRecoveryITCase.java:336)
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink issue #4415: [FLINK-7269] Refactor passing of dynamic properties

[zjureel]

Github user zjureel commented on the issue:

https://github.com/apache/flink/pull/4415
  
@tillrohrmann Great, use `loadConfigurationWithDynamicProperties` instead 
of `loadConfiguration` and `setDynamicProperties` will be more concise, it 
sounds good to me, thanks


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLINK-7269) Refactor passing of dynamic properties

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108302#comment-16108302
 ] 

ASF GitHub Bot commented on FLINK-7269:
---

Github user zjureel commented on the issue:

https://github.com/apache/flink/pull/4415
  
@tillrohrmann Great, use `loadConfigurationWithDynamicProperties` instead 
of `loadConfiguration` and `setDynamicProperties` will be more concise, it 
sounds good to me, thanks


> Refactor passing of dynamic properties
> --
>
> Key: FLINK-7269
> URL: https://issues.apache.org/jira/browse/FLINK-7269
> Project: Flink
>  Issue Type: Improvement
>  Components: Configuration
>Affects Versions: 1.3.1
>Reporter: Till Rohrmann
>Assignee: Fang Yong
>
> In order to set dynamic properties when loading the {{Configuration}} via 
> {{GlobalConfiguration.loadConfiguration}}, we currently set a static field in 
> {{GlobalConfiguration}} which is read whenever we load the {{Configuration}}.
> I think this is not a good pattern I propose to remove this functionality. 
> Instead we should explicitly add the dynamic properties to the loaded 
> {{Configuration}} at start of the application.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7302) Failed to run CorrelateITCase class under windows environment

[mingleizhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108252#comment-16108252
 ] 

mingleizhang commented on FLINK-7302:
-

If we dont fix this issue. I guess all windows user can not run some tests, 
like {{Kafka010ITCase}} I found today. It will encounter a compilation error.

> Failed to run CorrelateITCase class under windows environment
> -
>
> Key: FLINK-7302
> URL: https://issues.apache.org/jira/browse/FLINK-7302
> Project: Flink
>  Issue Type: Bug
>  Components: Table API & SQL
> Environment: Windows 7 
>Reporter: mingleizhang
>Assignee: mingleizhang
>
> Error:(220, 5) reference to UserDefinedFunctionTestUtils is ambiguous;
> it is imported twice in the same scope by
> import org.apache.flink.table.utils._
> and import org.apache.flink.table.runtime.utils._
> UserDefinedFunctionTestUtils.setJobParameters(env, Map("word_separator" 
> -> "#"))
> Error:(240, 5) reference to UserDefinedFunctionTestUtils is ambiguous;
> it is imported twice in the same scope by
> import org.apache.flink.table.utils._
> and import org.apache.flink.table.runtime.utils._
> UserDefinedFunctionTestUtils.setJobParameters(
> Error:(107, 5) reference to UserDefinedFunctionTestUtils is ambiguous;
> it is imported twice in the same scope by
> import org.apache.flink.table.utils._
> and import org.apache.flink.table.runtime.utils._
> UserDefinedFunctionTestUtils.setJobParameters(env, Map("word_separator" 
> -> " "))
> Error:(129, 5) reference to UserDefinedFunctionTestUtils is ambiguous;
> it is imported twice in the same scope by
> import org.apache.flink.table.utils._
> and import org.apache.flink.table.runtime.utils._
> UserDefinedFunctionTestUtils.setJobParameters(



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7299) Write GenericRecord using AvroOutputFormat

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108207#comment-16108207
 ] 

ASF GitHub Bot commented on FLINK-7299:
---

Github user soniclavier commented on the issue:

https://github.com/apache/flink/pull/4422
  
Link to passed Travis build : 
https://travis-ci.org/soniclavier/flink/builds/259243850


> Write GenericRecord using AvroOutputFormat
> --
>
> Key: FLINK-7299
> URL: https://issues.apache.org/jira/browse/FLINK-7299
> Project: Flink
>  Issue Type: Improvement
>  Components: Batch Connectors and Input/Output Formats
>Reporter: Vishnu Viswanath
>Assignee: Vishnu Viswanath
>Priority: Minor
>
> Allow AvroOutputFormat to write GenericRecords



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink issue #4422: [FLINK-7299][AVRO] Write GenericRecord using AvroOutputFo...

[soniclavier]

Github user soniclavier commented on the issue:

https://github.com/apache/flink/pull/4422
  
Link to passed Travis build : 
https://travis-ci.org/soniclavier/flink/builds/259243850


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (FLINK-7330) Unable to read file on windows - InvalidFilePathException

[Mohit Anchlia (JIRA)]

Mohit Anchlia created FLINK-7330:


 Summary: Unable to read file on windows - InvalidFilePathException
 Key: FLINK-7330
 URL: https://issues.apache.org/jira/browse/FLINK-7330
 Project: Flink
  Issue Type: Bug
 Environment: windows 10, flink 1.3.1
Reporter: Mohit Anchlia


It doesn't matter what I use for inputFormat.setFilePath() I keep getting this 
error:

{code:java}
//doesn't work
format.setFilePath("file:///c:/proj/test/a.txt.txt");


format.setFilePath("file:/c:/proj/test/a.txt.txt");

{code}

{code:java}
Caused by: java.nio.file.InvalidPathException: Illegal char <:> at index 2: 
/C:/Users/m/default/flink-example/pom.xml

at sun.nio.fs.WindowsPathParser.normalize(Unknown Source)

at sun.nio.fs.WindowsPathParser.parse(Unknown Source)

at sun.nio.fs.WindowsPathParser.parse(Unknown Source)

{code}




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (FLINK-7329) Unable to use FileInputStream with StreamExecutionEnvironment.readFile with interval

[Mohit Anchlia (JIRA)]

Mohit Anchlia created FLINK-7329:


 Summary: Unable to use FileInputStream with 
StreamExecutionEnvironment.readFile with interval
 Key: FLINK-7329
 URL: https://issues.apache.org/jira/browse/FLINK-7329
 Project: Flink
  Issue Type: Bug
 Environment: redhat 6, flink 1.3.1
Reporter: Mohit Anchlia


{code:java}

FileInputFormat fileInputFormat = new 
TextInputFormat(new Path(args[0]));
fileInputFormat.setNestedFileEnumeration(true);

streamEnv.readFile(fileInputFormat, args[0],
FileProcessingMode.PROCESS_CONTINUOUSLY, 
1L).print();

{code}

When trying to run this code I get following error. It's not clear what the 
problem is here and why it fails with this message.

[main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - class 
org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit does 
not contain a setter for field modificationTime
[main] INFO org.apache.flink.api.java.typeutils.TypeExtractor - class 
org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit is 
not a valid POJO type because not all fields are valid POJO fields.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4355: [FLINK-7206] [table] Implementation of DataView to...

[fhueske]

Github user fhueske commented on a diff in the pull request:

https://github.com/apache/flink/pull/4355#discussion_r130461621
  
--- Diff: 
flink-libraries/flink-table/src/main/scala/org/apache/flink/table/codegen/AggregationCodeGenerator.scala
 ---
@@ -405,6 +481,17 @@ class AggregationCodeGenerator(
   }
 }
 
+val aggFuncCode = Seq(
+  genSetAggregationResults,
+  genAccumulate,
+  genRetract,
+  genCreateAccumulators,
+  genSetForwardedFields,
+  genSetConstantFlags,
+  genCreateOutputRow,
+  genMergeAccumulatorsPair,
+  genResetAccumulator).mkString("\n")
--- End diff --

I think we need an additional `cleanUp()` method that clears all state 
objects for the current key. Otherwise, we will have memory leaks. The 
`cleanUp()` method must be called when the state retention timers trigger.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #4355: [FLINK-7206] [table] Implementation of DataView to...

[fhueske]

Github user fhueske commented on a diff in the pull request:

https://github.com/apache/flink/pull/4355#discussion_r130459674
  
--- Diff: 
flink-libraries/flink-table/src/main/scala/org/apache/flink/table/codegen/AggregationCodeGenerator.scala
 ---
@@ -222,14 +271,22 @@ class AggregationCodeGenerator(
 j"""
|  public final void retract(
|org.apache.flink.types.Row accs,
-   |org.apache.flink.types.Row input)""".stripMargin
+   |org.apache.flink.types.Row input) throws Exception 
""".stripMargin
 
   val retract: String = {
-for (i <- aggs.indices) yield
+for (i <- aggs.indices) yield {
+  val setDataView = if (accConfig.isDefined && 
accConfig.get.isUseState) {
+genDataViewFieldSetter(s"acc$i", accConfig.get.accSpecs(i))
--- End diff --

Same as for `setAggregationResults()`. I think we should reuse the state 
objects.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #4355: [FLINK-7206] [table] Implementation of DataView to...

[fhueske]

Github user fhueske commented on a diff in the pull request:

https://github.com/apache/flink/pull/4355#discussion_r130458506
  
--- Diff: 
flink-libraries/flink-table/src/main/scala/org/apache/flink/table/codegen/AggregationCodeGenerator.scala
 ---
@@ -179,13 +214,19 @@ class AggregationCodeGenerator(
|  ${aggMapping(i)},
|  (${accTypes(i)}) accs.getField($i));""".stripMargin
   } else {
+val setDataView = if (accConfig.isDefined && 
accConfig.get.isUseState) {
--- End diff --

This will create new `MapState` (or `ListState`) objects in every 
invocation of `setAggregationResults()`. I think we can make the state objects 
members of the `GeneratedAggregations` class and just set them into the 
accumulator.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLINK-7206) Implementation of DataView to support state access for UDAGG

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107954#comment-16107954
 ] 

ASF GitHub Bot commented on FLINK-7206:
---

Github user fhueske commented on a diff in the pull request:

https://github.com/apache/flink/pull/4355#discussion_r130459674
  
--- Diff: 
flink-libraries/flink-table/src/main/scala/org/apache/flink/table/codegen/AggregationCodeGenerator.scala
 ---
@@ -222,14 +271,22 @@ class AggregationCodeGenerator(
 j"""
|  public final void retract(
|org.apache.flink.types.Row accs,
-   |org.apache.flink.types.Row input)""".stripMargin
+   |org.apache.flink.types.Row input) throws Exception 
""".stripMargin
 
   val retract: String = {
-for (i <- aggs.indices) yield
+for (i <- aggs.indices) yield {
+  val setDataView = if (accConfig.isDefined && 
accConfig.get.isUseState) {
+genDataViewFieldSetter(s"acc$i", accConfig.get.accSpecs(i))
--- End diff --

Same as for `setAggregationResults()`. I think we should reuse the state 
objects.


> Implementation of DataView to support state access for UDAGG
> 
>
> Key: FLINK-7206
> URL: https://issues.apache.org/jira/browse/FLINK-7206
> Project: Flink
>  Issue Type: Sub-task
>  Components: Table API & SQL
>Reporter: Kaibo Zhou
>Assignee: Kaibo Zhou
>
> Implementation of MapView and ListView to support state access for UDAGG.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7206) Implementation of DataView to support state access for UDAGG

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107950#comment-16107950
 ] 

ASF GitHub Bot commented on FLINK-7206:
---

Github user fhueske commented on a diff in the pull request:

https://github.com/apache/flink/pull/4355#discussion_r130459631
  
--- Diff: 
flink-libraries/flink-table/src/main/scala/org/apache/flink/table/codegen/AggregationCodeGenerator.scala
 ---
@@ -201,14 +242,22 @@ class AggregationCodeGenerator(
 j"""
|  public final void accumulate(
|org.apache.flink.types.Row accs,
-   |org.apache.flink.types.Row input)""".stripMargin
+   |org.apache.flink.types.Row input) throws Exception 
""".stripMargin
 
   val accumulate: String = {
-for (i <- aggs.indices) yield
+for (i <- aggs.indices) yield {
+  val setDataView = if (accConfig.isDefined && 
accConfig.get.isUseState) {
+genDataViewFieldSetter(s"acc$i", accConfig.get.accSpecs(i))
--- End diff --

Same as for `setAggregationResults()`. I think we should reuse the state 
objects.


> Implementation of DataView to support state access for UDAGG
> 
>
> Key: FLINK-7206
> URL: https://issues.apache.org/jira/browse/FLINK-7206
> Project: Flink
>  Issue Type: Sub-task
>  Components: Table API & SQL
>Reporter: Kaibo Zhou
>Assignee: Kaibo Zhou
>
> Implementation of MapView and ListView to support state access for UDAGG.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7206) Implementation of DataView to support state access for UDAGG

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107952#comment-16107952
 ] 

ASF GitHub Bot commented on FLINK-7206:
---

Github user fhueske commented on a diff in the pull request:

https://github.com/apache/flink/pull/4355#discussion_r130461181
  
--- Diff: 
flink-libraries/flink-table/src/main/scala/org/apache/flink/table/codegen/AggregationCodeGenerator.scala
 ---
@@ -386,13 +455,20 @@ class AggregationCodeGenerator(
   val sig: String =
 j"""
|  public final void resetAccumulator(
-   |org.apache.flink.types.Row accs)""".stripMargin
+   |org.apache.flink.types.Row accs) throws Exception 
""".stripMargin
 
   val reset: String = {
-for (i <- aggs.indices) yield
+for (i <- aggs.indices) yield {
+  val setDataView = if (accConfig.isDefined && 
accConfig.get.isUseState) {
+genDataViewFieldSetter(s"acc$i", accConfig.get.accSpecs(i))
--- End diff --

Should reuse the state objects.


> Implementation of DataView to support state access for UDAGG
> 
>
> Key: FLINK-7206
> URL: https://issues.apache.org/jira/browse/FLINK-7206
> Project: Flink
>  Issue Type: Sub-task
>  Components: Table API & SQL
>Reporter: Kaibo Zhou
>Assignee: Kaibo Zhou
>
> Implementation of MapView and ListView to support state access for UDAGG.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4355: [FLINK-7206] [table] Implementation of DataView to...

[fhueske]

Github user fhueske commented on a diff in the pull request:

https://github.com/apache/flink/pull/4355#discussion_r130461181
  
--- Diff: 
flink-libraries/flink-table/src/main/scala/org/apache/flink/table/codegen/AggregationCodeGenerator.scala
 ---
@@ -386,13 +455,20 @@ class AggregationCodeGenerator(
   val sig: String =
 j"""
|  public final void resetAccumulator(
-   |org.apache.flink.types.Row accs)""".stripMargin
+   |org.apache.flink.types.Row accs) throws Exception 
""".stripMargin
 
   val reset: String = {
-for (i <- aggs.indices) yield
+for (i <- aggs.indices) yield {
+  val setDataView = if (accConfig.isDefined && 
accConfig.get.isUseState) {
+genDataViewFieldSetter(s"acc$i", accConfig.get.accSpecs(i))
--- End diff --

Should reuse the state objects.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLINK-7206) Implementation of DataView to support state access for UDAGG

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107953#comment-16107953
 ] 

ASF GitHub Bot commented on FLINK-7206:
---

Github user fhueske commented on a diff in the pull request:

https://github.com/apache/flink/pull/4355#discussion_r130461621
  
--- Diff: 
flink-libraries/flink-table/src/main/scala/org/apache/flink/table/codegen/AggregationCodeGenerator.scala
 ---
@@ -405,6 +481,17 @@ class AggregationCodeGenerator(
   }
 }
 
+val aggFuncCode = Seq(
+  genSetAggregationResults,
+  genAccumulate,
+  genRetract,
+  genCreateAccumulators,
+  genSetForwardedFields,
+  genSetConstantFlags,
+  genCreateOutputRow,
+  genMergeAccumulatorsPair,
+  genResetAccumulator).mkString("\n")
--- End diff --

I think we need an additional `cleanUp()` method that clears all state 
objects for the current key. Otherwise, we will have memory leaks. The 
`cleanUp()` method must be called when the state retention timers trigger.


> Implementation of DataView to support state access for UDAGG
> 
>
> Key: FLINK-7206
> URL: https://issues.apache.org/jira/browse/FLINK-7206
> Project: Flink
>  Issue Type: Sub-task
>  Components: Table API & SQL
>Reporter: Kaibo Zhou
>Assignee: Kaibo Zhou
>
> Implementation of MapView and ListView to support state access for UDAGG.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7206) Implementation of DataView to support state access for UDAGG

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107951#comment-16107951
 ] 

ASF GitHub Bot commented on FLINK-7206:
---

Github user fhueske commented on a diff in the pull request:

https://github.com/apache/flink/pull/4355#discussion_r130458506
  
--- Diff: 
flink-libraries/flink-table/src/main/scala/org/apache/flink/table/codegen/AggregationCodeGenerator.scala
 ---
@@ -179,13 +214,19 @@ class AggregationCodeGenerator(
|  ${aggMapping(i)},
|  (${accTypes(i)}) accs.getField($i));""".stripMargin
   } else {
+val setDataView = if (accConfig.isDefined && 
accConfig.get.isUseState) {
--- End diff --

This will create new `MapState` (or `ListState`) objects in every 
invocation of `setAggregationResults()`. I think we can make the state objects 
members of the `GeneratedAggregations` class and just set them into the 
accumulator.


> Implementation of DataView to support state access for UDAGG
> 
>
> Key: FLINK-7206
> URL: https://issues.apache.org/jira/browse/FLINK-7206
> Project: Flink
>  Issue Type: Sub-task
>  Components: Table API & SQL
>Reporter: Kaibo Zhou
>Assignee: Kaibo Zhou
>
> Implementation of MapView and ListView to support state access for UDAGG.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4355: [FLINK-7206] [table] Implementation of DataView to...

[fhueske]

Github user fhueske commented on a diff in the pull request:

https://github.com/apache/flink/pull/4355#discussion_r130459631
  
--- Diff: 
flink-libraries/flink-table/src/main/scala/org/apache/flink/table/codegen/AggregationCodeGenerator.scala
 ---
@@ -201,14 +242,22 @@ class AggregationCodeGenerator(
 j"""
|  public final void accumulate(
|org.apache.flink.types.Row accs,
-   |org.apache.flink.types.Row input)""".stripMargin
+   |org.apache.flink.types.Row input) throws Exception 
""".stripMargin
 
   val accumulate: String = {
-for (i <- aggs.indices) yield
+for (i <- aggs.indices) yield {
+  val setDataView = if (accConfig.isDefined && 
accConfig.get.isUseState) {
+genDataViewFieldSetter(s"acc$i", accConfig.get.accSpecs(i))
--- End diff --

Same as for `setAggregationResults()`. I think we should reuse the state 
objects.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLINK-7223) Increase DEFAULT_SHARD_DISCOVERY_INTERVAL_MILLIS for Flink-kinesis-connector

[Stephan Ewen (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107888#comment-16107888
 ] 

Stephan Ewen commented on FLINK-7223:
-

Hmmm... Is there a way to auto-configure this value as in the following way:
  - The Flink job would theoretically do discovery once per 5 seconds (this 
means running up to 50 Flink jobs er account)
  - The interval in which the TaskManagers can discover is than {{5 seconds x 
source parallelism}}


> Increase DEFAULT_SHARD_DISCOVERY_INTERVAL_MILLIS for Flink-kinesis-connector
> 
>
> Key: FLINK-7223
> URL: https://issues.apache.org/jira/browse/FLINK-7223
> Project: Flink
>  Issue Type: Improvement
>  Components: Kinesis Connector
>Affects Versions: 1.3.0
>Reporter: Bowen Li
>Assignee: Bowen Li
>Priority: Minor
> Fix For: 1.4.0
>
>
> Background: {{DEFAULT_SHARD_DISCOVERY_INTERVAL_MILLIS}} in 
> {{org.apache.flink.streaming.connectors.kinesis.config.ConsumerConfigConstants}}
>  is the default value for Flink to call Kinesis's {{describeStream()}} API.
> Problem: Right now, its value is 10,000millis (10sec), which is too short. We 
> ran into problems that Flink-kinesis-connector's call of {{describeStream()}} 
> exceeds Kinesis rate limit, and broken Flink taskmanager.
> According to 
> http://docs.aws.amazon.com/kinesis/latest/APIReference/API_DescribeStream.html,
>  
> "This operation has a limit of 10 transactions per second per account.". What 
> it means is that the 10transaction/account is a limit on a single 
> organization's AWS account..:(  We contacted AWS Support, and confirmed 
> this. If you have more applications (either other Flink apps or non-Flink 
> apps) competing aggressively with your Flink app on this API, your Flink app 
> breaks. 
> I propose increasing the value DEFAULT_SHARD_DISCOVERY_INTERVAL_MILLIS from 
> 10,000millis(10sec) to preferably 300,000 (5min). Or at least 60,000 (1min) 
> if anyone has a solid reason arguing that 5min is too long, 
> This is also related to https://issues.apache.org/jira/browse/FLINK-6365



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (FLINK-7242) Drop Java 7 Support

[Chesnay Schepler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLINK-7242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chesnay Schepler reassigned FLINK-7242:
---

Assignee: Chesnay Schepler

> Drop Java 7 Support
> ---
>
> Key: FLINK-7242
> URL: https://issues.apache.org/jira/browse/FLINK-7242
> Project: Flink
>  Issue Type: Task
>  Components: Build System
>Reporter: Eron Wright 
>Assignee: Chesnay Schepler
>Priority: Critical
> Fix For: 1.4.0
>
>
> This is the umbrella issue for dropping Java 7 support.   The decision was 
> taken following a vote 
> [here|http://mail-archives.apache.org/mod_mbox/flink-dev/201707.mbox/%3CCANC1h_tawd90CU12v%2BfQ%2BQU2ORsh%3Dnob7AehT11jGHs1g5Hqtg%40mail.gmail.com%3E]
>  and announced 
> [here|http://mail-archives.apache.org/mod_mbox/flink-dev/201707.mbox/%3CCANC1h_vnxpiBnAB0OmQPD6NMH6L_PLCyWYsX32mZ0H%2BXP3%2BheQ%40mail.gmail.com%3E].
>   Reasons cited include new language features and compatibility with Akka 2.4 
> and Scala 2.12.
> Please open sub-tasks as necessary.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (FLINK-7242) Drop Java 7 Support

[Chesnay Schepler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLINK-7242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chesnay Schepler reassigned FLINK-7242:
---

Assignee: Chesnay Schepler

> Drop Java 7 Support
> ---
>
> Key: FLINK-7242
> URL: https://issues.apache.org/jira/browse/FLINK-7242
> Project: Flink
>  Issue Type: Task
>  Components: Build System
>Reporter: Eron Wright 
>Assignee: Chesnay Schepler
>Priority: Critical
> Fix For: 1.4.0
>
>
> This is the umbrella issue for dropping Java 7 support.   The decision was 
> taken following a vote 
> [here|http://mail-archives.apache.org/mod_mbox/flink-dev/201707.mbox/%3CCANC1h_tawd90CU12v%2BfQ%2BQU2ORsh%3Dnob7AehT11jGHs1g5Hqtg%40mail.gmail.com%3E]
>  and announced 
> [here|http://mail-archives.apache.org/mod_mbox/flink-dev/201707.mbox/%3CCANC1h_vnxpiBnAB0OmQPD6NMH6L_PLCyWYsX32mZ0H%2BXP3%2BheQ%40mail.gmail.com%3E].
>   Reasons cited include new language features and compatibility with Akka 2.4 
> and Scala 2.12.
> Please open sub-tasks as necessary.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (FLINK-7242) Drop Java 7 Support

[Chesnay Schepler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLINK-7242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chesnay Schepler reassigned FLINK-7242:
---

Assignee: (was: Chesnay Schepler)

> Drop Java 7 Support
> ---
>
> Key: FLINK-7242
> URL: https://issues.apache.org/jira/browse/FLINK-7242
> Project: Flink
>  Issue Type: Task
>  Components: Build System
>Reporter: Eron Wright 
>Priority: Critical
> Fix For: 1.4.0
>
>
> This is the umbrella issue for dropping Java 7 support.   The decision was 
> taken following a vote 
> [here|http://mail-archives.apache.org/mod_mbox/flink-dev/201707.mbox/%3CCANC1h_tawd90CU12v%2BfQ%2BQU2ORsh%3Dnob7AehT11jGHs1g5Hqtg%40mail.gmail.com%3E]
>  and announced 
> [here|http://mail-archives.apache.org/mod_mbox/flink-dev/201707.mbox/%3CCANC1h_vnxpiBnAB0OmQPD6NMH6L_PLCyWYsX32mZ0H%2BXP3%2BheQ%40mail.gmail.com%3E].
>   Reasons cited include new language features and compatibility with Akka 2.4 
> and Scala 2.12.
> Please open sub-tasks as necessary.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7328) Remove Flink's futures from SlotManager

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107844#comment-16107844
 ] 

ASF GitHub Bot commented on FLINK-7328:
---

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4443

[FLINK-7328] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in SlotManager

## What is the purpose of the change

Replace Flink's futures with Java 8's CompletableFuture in SlotManager.

This PR is based on #4438.

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)

## Documentation

  - Does this pull request introduce a new feature? (no)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink rfSlotManager

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4443.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4443


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit c265fa386184b1fb940187c7f97c14e1542509de
Author: Till Rohrmann 
Date:   2017-07-31T17:35:14Z

[FLINK-7324] [futures] Replace Flink's future with Java 8's 
CompletableFuture in SlotPool

commit 5ff0bfdf9c7e9dc5a6411986eeb7f444763d8249
Author: Till Rohrmann 
Date:   2017-07-31T19:38:28Z

[FLINK-7328] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in SlotManager




> Remove Flink's futures from SlotManager
> ---
>
> Key: FLINK-7328
> URL: https://issues.apache.org/jira/browse/FLINK-7328
> Project: Flink
>  Issue Type: Sub-task
>  Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.4.0
>
>
> This only affects the internally used Flink futures.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4443: [FLINK-7328] [futures] Replace Flink's futures wit...

[tillrohrmann]

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4443

[FLINK-7328] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in SlotManager

## What is the purpose of the change

Replace Flink's futures with Java 8's CompletableFuture in SlotManager.

This PR is based on #4438.

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)

## Documentation

  - Does this pull request introduce a new feature? (no)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink rfSlotManager

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4443.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4443


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit c265fa386184b1fb940187c7f97c14e1542509de
Author: Till Rohrmann 
Date:   2017-07-31T17:35:14Z

[FLINK-7324] [futures] Replace Flink's future with Java 8's 
CompletableFuture in SlotPool

commit 5ff0bfdf9c7e9dc5a6411986eeb7f444763d8249
Author: Till Rohrmann 
Date:   2017-07-31T19:38:28Z

[FLINK-7328] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in SlotManager




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (FLINK-7328) Remove Flink's futures from SlotManager

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-7328:


 Summary: Remove Flink's futures from SlotManager
 Key: FLINK-7328
 URL: https://issues.apache.org/jira/browse/FLINK-7328
 Project: Flink
  Issue Type: Sub-task
  Components: Distributed Coordination
Affects Versions: 1.4.0
Reporter: Till Rohrmann
Assignee: Till Rohrmann
Priority: Minor


This only affects the internally used Flink futures.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7327) Remove Flink's futures from StreamRecordQueueEntry

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107839#comment-16107839
 ] 

ASF GitHub Bot commented on FLINK-7327:
---

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4442

[FLINK-7327] [futures] Replace Flink's future with Java 8's 
CompletableFuture in StreamRecordQueueEntry

## What is the purpose of the change

Replace Flink's future with Java 8's CompletableFuture in 
StreamRecordQueueEntry.

This PR is based on #4429.

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)

## Documentation

  - Does this pull request introduce a new feature? (no)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink rfWatermarkQueryEntry

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4442.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4442


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit a192ebbec44c2a5900cfba3ebc603d7afe271c39
Author: Till Rohrmann 
Date:   2017-07-31T19:31:26Z

[FLINK-7327] [futures] Replace Flink's future with Java 8's 
CompletableFuture in StreamRecordQueueEntry




> Remove Flink's futures from StreamRecordQueueEntry
> --
>
> Key: FLINK-7327
> URL: https://issues.apache.org/jira/browse/FLINK-7327
> Project: Flink
>  Issue Type: Sub-task
>  Components: Local Runtime
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4442: [FLINK-7327] [futures] Replace Flink's future with...

[tillrohrmann]

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4442

[FLINK-7327] [futures] Replace Flink's future with Java 8's 
CompletableFuture in StreamRecordQueueEntry

## What is the purpose of the change

Replace Flink's future with Java 8's CompletableFuture in 
StreamRecordQueueEntry.

This PR is based on #4429.

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)

## Documentation

  - Does this pull request introduce a new feature? (no)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink rfWatermarkQueryEntry

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4442.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4442


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit a192ebbec44c2a5900cfba3ebc603d7afe271c39
Author: Till Rohrmann 
Date:   2017-07-31T19:31:26Z

[FLINK-7327] [futures] Replace Flink's future with Java 8's 
CompletableFuture in StreamRecordQueueEntry




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (FLINK-7327) Remove Flink's futures from StreamRecordQueueEntry

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-7327:


 Summary: Remove Flink's futures from StreamRecordQueueEntry
 Key: FLINK-7327
 URL: https://issues.apache.org/jira/browse/FLINK-7327
 Project: Flink
  Issue Type: Sub-task
  Components: Local Runtime
Affects Versions: 1.4.0
Reporter: Till Rohrmann
Assignee: Till Rohrmann
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7326) Remove Flink's futures from RegisteredRpcConnection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107770#comment-16107770
 ] 

ASF GitHub Bot commented on FLINK-7326:
---

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4440#discussion_r130429986
  
--- Diff: 
flink-runtime/src/main/java/org/apache/flink/runtime/registration/RegisteredRpcConnection.java
 ---
@@ -88,24 +86,17 @@ public void start() {
pendingRegistration = checkNotNull(generateRegistration());
pendingRegistration.startRegistration();
 
-   Future> future = 
pendingRegistration.getFuture();
-
-   Future registrationSuccessFuture = 
future.thenAcceptAsync(new AcceptFunction>() {
-   @Override
-   public void accept(Tuple2 result) {
-   targetGateway = result.f0;
-   onRegistrationSuccess(result.f1);
-   }
-   }, executor);
-
-   // this future should only ever fail if there is a bug, not if 
the registration is declined
-   registrationSuccessFuture.exceptionallyAsync(new 
ApplyFunction() {
-   @Override
-   public Void apply(Throwable failure) {
-   onRegistrationFailure(failure);
-   return null;
-   }
-   }, executor);
+   CompletableFuture> future = 
pendingRegistration.getFuture();
+
+   future.whenCompleteAsync(
+   (Tuple2 result, Throwable failure) -> 
{
+   if (failure != null) {
--- End diff --

I'm wondering whether we should revert the condition so that the 
"successful" path comes first.


> Remove Flink's futures from RegisteredRpcConnection
> ---
>
> Key: FLINK-7326
> URL: https://issues.apache.org/jira/browse/FLINK-7326
> Project: Flink
>  Issue Type: Sub-task
>  Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7326) Remove Flink's futures from RegisteredRpcConnection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107769#comment-16107769
 ] 

ASF GitHub Bot commented on FLINK-7326:
---

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4440#discussion_r130429770
  
--- Diff: 
flink-runtime/src/main/java/org/apache/flink/runtime/registration/RegisteredRpcConnection.java
 ---
@@ -88,24 +86,17 @@ public void start() {
pendingRegistration = checkNotNull(generateRegistration());
pendingRegistration.startRegistration();
 
-   Future> future = 
pendingRegistration.getFuture();
-
-   Future registrationSuccessFuture = 
future.thenAcceptAsync(new AcceptFunction>() {
-   @Override
-   public void accept(Tuple2 result) {
-   targetGateway = result.f0;
-   onRegistrationSuccess(result.f1);
-   }
-   }, executor);
-
-   // this future should only ever fail if there is a bug, not if 
the registration is declined
--- End diff --

We could keep this comment, or is it no longer valid?


> Remove Flink's futures from RegisteredRpcConnection
> ---
>
> Key: FLINK-7326
> URL: https://issues.apache.org/jira/browse/FLINK-7326
> Project: Flink
>  Issue Type: Sub-task
>  Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4440: [FLINK-7326] [futures] Replace Flink's future with...

[zentol]

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4440#discussion_r130429770
  
--- Diff: 
flink-runtime/src/main/java/org/apache/flink/runtime/registration/RegisteredRpcConnection.java
 ---
@@ -88,24 +86,17 @@ public void start() {
pendingRegistration = checkNotNull(generateRegistration());
pendingRegistration.startRegistration();
 
-   Future> future = 
pendingRegistration.getFuture();
-
-   Future registrationSuccessFuture = 
future.thenAcceptAsync(new AcceptFunction>() {
-   @Override
-   public void accept(Tuple2 result) {
-   targetGateway = result.f0;
-   onRegistrationSuccess(result.f1);
-   }
-   }, executor);
-
-   // this future should only ever fail if there is a bug, not if 
the registration is declined
--- End diff --

We could keep this comment, or is it no longer valid?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #4440: [FLINK-7326] [futures] Replace Flink's future with...

[zentol]

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4440#discussion_r130429986
  
--- Diff: 
flink-runtime/src/main/java/org/apache/flink/runtime/registration/RegisteredRpcConnection.java
 ---
@@ -88,24 +86,17 @@ public void start() {
pendingRegistration = checkNotNull(generateRegistration());
pendingRegistration.startRegistration();
 
-   Future> future = 
pendingRegistration.getFuture();
-
-   Future registrationSuccessFuture = 
future.thenAcceptAsync(new AcceptFunction>() {
-   @Override
-   public void accept(Tuple2 result) {
-   targetGateway = result.f0;
-   onRegistrationSuccess(result.f1);
-   }
-   }, executor);
-
-   // this future should only ever fail if there is a bug, not if 
the registration is declined
-   registrationSuccessFuture.exceptionallyAsync(new 
ApplyFunction() {
-   @Override
-   public Void apply(Throwable failure) {
-   onRegistrationFailure(failure);
-   return null;
-   }
-   }, executor);
+   CompletableFuture> future = 
pendingRegistration.getFuture();
+
+   future.whenCompleteAsync(
+   (Tuple2 result, Throwable failure) -> 
{
+   if (failure != null) {
--- End diff --

I'm wondering whether we should revert the condition so that the 
"successful" path comes first.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLINK-7324) Remove Flink's futures from SlotPool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107766#comment-16107766
 ] 

ASF GitHub Bot commented on FLINK-7324:
---

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4438#discussion_r130429317
  
--- Diff: 
flink-runtime/src/main/java/org/apache/flink/runtime/concurrent/FutureUtils.java
 ---
@@ -344,4 +344,26 @@ public void onComplete(Throwable failure, T success) 
throws Throwable {
 
return result;
}
+
+   /**
+* Converts a Java 8 {@link java.util.concurrent.CompletableFuture} 
into a Flink {@link Future}.
+*
+* @param javaFuture to convert to a Flink future
+* @param  type of the future value
+* @return Flink future
+*/
+   public static  Future 
toFlinkFuture(java.util.concurrent.CompletableFuture javaFuture) {
--- End diff --

Add deprecated annotation?


> Remove Flink's futures from SlotPool
> 
>
> Key: FLINK-7324
> URL: https://issues.apache.org/jira/browse/FLINK-7324
> Project: Flink
>  Issue Type: Sub-task
>  Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.4.0
>
>
> This only refers to the internal usage of Flink's {{Future}} usage in 
> {{SlotPool}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4438: [FLINK-7324] [futures] Replace Flink's future with...

[zentol]

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4438#discussion_r130429317
  
--- Diff: 
flink-runtime/src/main/java/org/apache/flink/runtime/concurrent/FutureUtils.java
 ---
@@ -344,4 +344,26 @@ public void onComplete(Throwable failure, T success) 
throws Throwable {
 
return result;
}
+
+   /**
+* Converts a Java 8 {@link java.util.concurrent.CompletableFuture} 
into a Flink {@link Future}.
+*
+* @param javaFuture to convert to a Flink future
+* @param  type of the future value
+* @return Flink future
+*/
+   public static  Future 
toFlinkFuture(java.util.concurrent.CompletableFuture javaFuture) {
--- End diff --

Add deprecated annotation?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLINK-7320) Remove Flink's futures from Scheduler

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107760#comment-16107760
 ] 

ASF GitHub Bot commented on FLINK-7320:
---

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4435#discussion_r130428135
  
--- Diff: 
flink-runtime/src/main/java/org/apache/flink/runtime/jobmanager/scheduler/Scheduler.java
 ---
@@ -152,12 +150,14 @@ else if (ret instanceof Future) {
}
}
catch (NoResourceAvailableException e) {
-   return FlinkCompletableFuture.completedExceptionally(e);
+   CompletableFuture notEnoughResources = new 
CompletableFuture<>();
--- End diff --

How about introducing a utility method to create an exceptionally completed 
`CompletableFuture` in one line?


> Remove Flink's futures from Scheduler
> -
>
> Key: FLINK-7320
> URL: https://issues.apache.org/jira/browse/FLINK-7320
> Project: Flink
>  Issue Type: Sub-task
>  Components: Scheduler
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4435: [FLINK-7320] [futures] Replace Flink's futures wit...

[zentol]

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4435#discussion_r130428135
  
--- Diff: 
flink-runtime/src/main/java/org/apache/flink/runtime/jobmanager/scheduler/Scheduler.java
 ---
@@ -152,12 +150,14 @@ else if (ret instanceof Future) {
}
}
catch (NoResourceAvailableException e) {
-   return FlinkCompletableFuture.completedExceptionally(e);
+   CompletableFuture notEnoughResources = new 
CompletableFuture<>();
--- End diff --

How about introducing a utility method to create an exceptionally completed 
`CompletableFuture` in one line?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLINK-7321) Remove Flink's futures from HeartbeatManager

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107759#comment-16107759
 ] 

ASF GitHub Bot commented on FLINK-7321:
---

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4434#discussion_r130427767
  
--- Diff: 
flink-runtime/src/test/java/org/apache/flink/runtime/heartbeat/HeartbeatManagerTest.java
 ---
@@ -288,7 +286,11 @@ public void testTargetUnmonitoring() throws 
InterruptedException, ExecutionExcep
this.payload = payload;
}
 
-   public Future getTimeoutFuture() {
+   public CompletableFuture getFuture() {
+   return future;
--- End diff --

this method appears to be identical to `getTimeoutFuture`.


> Remove Flink's futures from HeartbeatManager
> 
>
> Key: FLINK-7321
> URL: https://issues.apache.org/jira/browse/FLINK-7321
> Project: Flink
>  Issue Type: Sub-task
>  Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4434: [FLINK-7321] [futures] Replace Flink's futures wit...

[zentol]

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4434#discussion_r130427767
  
--- Diff: 
flink-runtime/src/test/java/org/apache/flink/runtime/heartbeat/HeartbeatManagerTest.java
 ---
@@ -288,7 +286,11 @@ public void testTargetUnmonitoring() throws 
InterruptedException, ExecutionExcep
this.payload = payload;
}
 
-   public Future getTimeoutFuture() {
+   public CompletableFuture getFuture() {
+   return future;
--- End diff --

this method appears to be identical to `getTimeoutFuture`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLINK-7317) Remove Flink's futures from ExecutionGraph

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107754#comment-16107754
 ] 

ASF GitHub Bot commented on FLINK-7317:
---

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4433#discussion_r130400792
  
--- Diff: 
flink-runtime/src/main/java/org/apache/flink/runtime/concurrent/FutureUtils.java
 ---
@@ -50,42 +49,38 @@
 * @param  type of the result
 * @return Future containing either the result of the operation or a 
{@link RetryException}
 */
-   public static  Future retry(
-   final Callable operation,
+   public static  java.util.concurrent.CompletableFuture retry(
+   final Callable 
operation,
final int retries,
final Executor executor) {
 
-   Future operationResultFuture;
+   java.util.concurrent.CompletableFuture operationResultFuture;
 
try {
operationResultFuture = operation.call();
} catch (Exception e) {
-   return FlinkCompletableFuture.completedExceptionally(
-   new RetryException("Could not execute the 
provided operation.", e));
+   java.util.concurrent.CompletableFuture 
exceptionResult = new java.util.concurrent.CompletableFuture<>();
+   exceptionResult.completeExceptionally(new 
RetryException("Could not execute the provided operation.", e));
+   return exceptionResult;
}
 
-   return operationResultFuture.handleAsync(new BiFunction() {
-   @Override
-   public Future apply(T t, Throwable throwable) {
+   return operationResultFuture.handleAsync(
+   (t, throwable) -> {
if (throwable != null) {
if (retries > 0) {
return retry(operation, retries 
- 1, executor);
} else {
-   return 
FlinkCompletableFuture.completedExceptionally(
-   new 
RetryException("Could not complete the operation. Number of retries " +
-   "has been 
exhausted.", throwable));
+   
java.util.concurrent.CompletableFuture exceptionResult = new 
java.util.concurrent.CompletableFuture<>();
+   
exceptionResult.completeExceptionally(new RetryException("Could not complete 
the operation. Number of retries " +
+   "has been exhausted.", 
throwable));
+   return exceptionResult;
}
} else {
-   return 
FlinkCompletableFuture.completed(t);
+   return 
java.util.concurrent.CompletableFuture.completedFuture(t);
}
-   }
-   }, executor)
-   .thenCompose(new ApplyFunction() {
-   @Override
-   public Future apply(Future value) {
-   return value;
-   }
-   });
+   },
+   executor)
+   .thenCompose(value -> value);
--- End diff --

what is this for?


> Remove Flink's futures from ExecutionGraph
> --
>
> Key: FLINK-7317
> URL: https://issues.apache.org/jira/browse/FLINK-7317
> Project: Flink
>  Issue Type: Sub-task
>  Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
> Fix For: 1.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7317) Remove Flink's futures from ExecutionGraph

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107755#comment-16107755
 ] 

ASF GitHub Bot commented on FLINK-7317:
---

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4433#discussion_r130401027
  
--- Diff: 
flink-runtime/src/main/java/org/apache/flink/runtime/concurrent/FutureUtils.java
 ---
@@ -199,25 +194,19 @@ public RetryException(Throwable cause) {
/** The function that is attached to all futures in the 
conjunction. Once a future
--- End diff --

javadoc seems outdated.


> Remove Flink's futures from ExecutionGraph
> --
>
> Key: FLINK-7317
> URL: https://issues.apache.org/jira/browse/FLINK-7317
> Project: Flink
>  Issue Type: Sub-task
>  Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
> Fix For: 1.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4433: [FLINK-7317] [futures] Replace Flink's futures wit...

[zentol]

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4433#discussion_r130400792
  
--- Diff: 
flink-runtime/src/main/java/org/apache/flink/runtime/concurrent/FutureUtils.java
 ---
@@ -50,42 +49,38 @@
 * @param  type of the result
 * @return Future containing either the result of the operation or a 
{@link RetryException}
 */
-   public static  Future retry(
-   final Callable operation,
+   public static  java.util.concurrent.CompletableFuture retry(
+   final Callable 
operation,
final int retries,
final Executor executor) {
 
-   Future operationResultFuture;
+   java.util.concurrent.CompletableFuture operationResultFuture;
 
try {
operationResultFuture = operation.call();
} catch (Exception e) {
-   return FlinkCompletableFuture.completedExceptionally(
-   new RetryException("Could not execute the 
provided operation.", e));
+   java.util.concurrent.CompletableFuture 
exceptionResult = new java.util.concurrent.CompletableFuture<>();
+   exceptionResult.completeExceptionally(new 
RetryException("Could not execute the provided operation.", e));
+   return exceptionResult;
}
 
-   return operationResultFuture.handleAsync(new BiFunction() {
-   @Override
-   public Future apply(T t, Throwable throwable) {
+   return operationResultFuture.handleAsync(
+   (t, throwable) -> {
if (throwable != null) {
if (retries > 0) {
return retry(operation, retries 
- 1, executor);
} else {
-   return 
FlinkCompletableFuture.completedExceptionally(
-   new 
RetryException("Could not complete the operation. Number of retries " +
-   "has been 
exhausted.", throwable));
+   
java.util.concurrent.CompletableFuture exceptionResult = new 
java.util.concurrent.CompletableFuture<>();
+   
exceptionResult.completeExceptionally(new RetryException("Could not complete 
the operation. Number of retries " +
+   "has been exhausted.", 
throwable));
+   return exceptionResult;
}
} else {
-   return 
FlinkCompletableFuture.completed(t);
+   return 
java.util.concurrent.CompletableFuture.completedFuture(t);
}
-   }
-   }, executor)
-   .thenCompose(new ApplyFunction() {
-   @Override
-   public Future apply(Future value) {
-   return value;
-   }
-   });
+   },
+   executor)
+   .thenCompose(value -> value);
--- End diff --

what is this for?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #4433: [FLINK-7317] [futures] Replace Flink's futures wit...

[zentol]

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4433#discussion_r130401027
  
--- Diff: 
flink-runtime/src/main/java/org/apache/flink/runtime/concurrent/FutureUtils.java
 ---
@@ -199,25 +194,19 @@ public RetryException(Throwable cause) {
/** The function that is attached to all futures in the 
conjunction. Once a future
--- End diff --

javadoc seems outdated.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink issue #4441: [FLINK-7301] [docs] Rework state documentation

[twalthr]

Github user twalthr commented on the issue:

https://github.com/apache/flink/pull/4441
  
CC @alpinegizmo 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLINK-7301) Rework state documentation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107732#comment-16107732
 ] 

ASF GitHub Bot commented on FLINK-7301:
---

Github user twalthr commented on the issue:

https://github.com/apache/flink/pull/4441
  
CC @alpinegizmo 


> Rework state documentation
> --
>
> Key: FLINK-7301
> URL: https://issues.apache.org/jira/browse/FLINK-7301
> Project: Flink
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> The documentation about state is spread across different pages, but this is 
> not consistent and it is hard to find what you need. I propose:
> "Mention State Backends and link to them in ""Streaming/Working with State"".
> Create category ""State & Fault Tolerance"" under ""Streaming"". Move 
> ""Working with State"", ""Checkpointing"" and ""Queryable State"".
> Move API related parts (90%) of ""Deployment/State & Fault Tolerance/State 
> Backends"" to ""Streaming/State & Fault Tolerance/State Backends"".
> Move all tuning things from ""Debugging/Large State"" to ""Deployment/State & 
> Fault Tolerance/State Backends"".
> Move ""Streaming/Working with State/Custom Serialization for Managed State"" 
> to ""Streaming/State & Fault Tolerance/Custom Serialization"" (Add a link 
> from previous position, also link from ""Data Types & Serialization"")."



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7301) Rework state documentation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107731#comment-16107731
 ] 

ASF GitHub Bot commented on FLINK-7301:
---

GitHub user twalthr opened a pull request:

https://github.com/apache/flink/pull/4441

[FLINK-7301] [docs] Rework state documentation

## What is the purpose of the change

*This PR restructures state related documentation pages. It introduces some 
state introduction page and moves some files (from `setup/` to `ops/`) 
according to the new documentation structure.*

## Brief change log

*Documentation changes only.*

## Verifying this change

*Built with built script and links checked.*

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): no
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: no
  - The serializers: no
  - The runtime per-record code paths (performance sensitive): no
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: no

## Documentation

  - Does this pull request introduce a new feature? no
  - If yes, how is the feature documented? not applicable



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/twalthr/flink FLINK-7301

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4441.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4441


commit b53e758216364903f211277052dbba4ae99da7d3
Author: twalthr 
Date:   2017-07-31T18:14:31Z

[FLINK-7301] [docs] Rework state documentation




> Rework state documentation
> --
>
> Key: FLINK-7301
> URL: https://issues.apache.org/jira/browse/FLINK-7301
> Project: Flink
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> The documentation about state is spread across different pages, but this is 
> not consistent and it is hard to find what you need. I propose:
> "Mention State Backends and link to them in ""Streaming/Working with State"".
> Create category ""State & Fault Tolerance"" under ""Streaming"". Move 
> ""Working with State"", ""Checkpointing"" and ""Queryable State"".
> Move API related parts (90%) of ""Deployment/State & Fault Tolerance/State 
> Backends"" to ""Streaming/State & Fault Tolerance/State Backends"".
> Move all tuning things from ""Debugging/Large State"" to ""Deployment/State & 
> Fault Tolerance/State Backends"".
> Move ""Streaming/Working with State/Custom Serialization for Managed State"" 
> to ""Streaming/State & Fault Tolerance/Custom Serialization"" (Add a link 
> from previous position, also link from ""Data Types & Serialization"")."



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4441: [FLINK-7301] [docs] Rework state documentation

[twalthr]

GitHub user twalthr opened a pull request:

https://github.com/apache/flink/pull/4441

[FLINK-7301] [docs] Rework state documentation

## What is the purpose of the change

*This PR restructures state related documentation pages. It introduces some 
state introduction page and moves some files (from `setup/` to `ops/`) 
according to the new documentation structure.*

## Brief change log

*Documentation changes only.*

## Verifying this change

*Built with built script and links checked.*

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): no
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: no
  - The serializers: no
  - The runtime per-record code paths (performance sensitive): no
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: no

## Documentation

  - Does this pull request introduce a new feature? no
  - If yes, how is the feature documented? not applicable



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/twalthr/flink FLINK-7301

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4441.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4441


commit b53e758216364903f211277052dbba4ae99da7d3
Author: twalthr 
Date:   2017-07-31T18:14:31Z

[FLINK-7301] [docs] Rework state documentation




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Comment Edited] (FLINK-7294) mesos.resourcemanager.framework.role not working

[Eron Wright (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107719#comment-16107719
 ] 

Eron Wright  edited comment on FLINK-7294 at 7/31/17 6:18 PM:
--

It looks to be a code defect, that Flink is registering to be offered resources 
from a certain role, but doesn't properly accept them.

The fix would likely involve adding role information to the resources that are 
added to the `TaskInfo`.  The `scalar` helper method should be amended to take 
a `role` argument, whose value would be drawn from the `mesosConfiguration`.  
See:
[LaunchableMesosWorker|https://github.com/apache/flink/blob/release-1.3.1/flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/LaunchableMesosWorker.java#L197]



was (Author: eronwright):
It looks to be a code defect, that Flink is registering to be offered resources 
from a certain role, but doesn't properly accept them.

The fix would likely involve adding role information to the resources that are 
added to the `TaskInfo`.  The `scalar` helper method should be amended to take 
a `role` argument, whose value would be drawn from the `mesosConfiguration`.  
See:
[LaunchableMesosWorker|https://github.com/apache/flink/blob/master/flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/LaunchableMesosWorker.java#L195]


> mesos.resourcemanager.framework.role not working
> 
>
> Key: FLINK-7294
> URL: https://issues.apache.org/jira/browse/FLINK-7294
> Project: Flink
>  Issue Type: Bug
>  Components: Mesos
>Affects Versions: 1.3.1
>Reporter: Bhumika Bayani
>Priority: Critical
>
> I am using the above said setting in flink-conf.yaml
> e.g.
> mesos.resourcemanager.framework.role: mesos_role_tasks
> I see a flink-scheduler registered in mesos/frameworks tab with above said 
> role.
> But the scheduler fails to launch any tasks inspite of getting 
> resource-offers from mesos-agents with correct role.
> The error seen is:
> 2017-07-28 13:23:00,683 INFO  
> org.apache.flink.mesos.runtime.clusterframework.MesosFlinkResourceManager  - 
> Mesos task taskmanager-03768 failed, with a TaskManager in launch or 
> registration. State: TASK_ERROR Reason: REASON_TASK_INVALID (Task uses more 
> resources cpus(\*):1; mem(\*):1024; ports(\*):[4006-4007] than available 
> cpus(mesos_role_tasks):7.4; mem(mesos_role_tasks):45876; 
> ports(mesos_role_tasks):[4002-4129, 4131-4380, 4382-4809, 4811-4957, 
> 4959-4966, 4968-4979, 4981-5049, 31000-31196, 31198-31431, 31433-31607, 
> 31609-32000]; ephemeral_storage(mesos_role_tasks):37662; 
> efs_storage(mesos_role_tasks):8.79609e+12; disk(mesos_role_tasks):5115)
> The request is made for resources with * role. We do not have mesos running 
> anywhere with * role. Thus task manager never come up. 
> Am I missing any configuration?
> I am using flink version 1.3.1



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7294) mesos.resourcemanager.framework.role not working

[Eron Wright (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107719#comment-16107719
 ] 

Eron Wright  commented on FLINK-7294:
-

It looks to be a code defect, that Flink is registering to be offered resources 
from a certain role, but doesn't properly accept them.

The fix would likely involve adding role information to the resources that are 
added to the `TaskInfo`.  The `scalar` helper method should be amended to take 
a `role` argument, whose value would be drawn from the `mesosConfiguration`.  
See:
[LaunchableMesosWorker|https://github.com/apache/flink/blob/master/flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/LaunchableMesosWorker.java#L195]


> mesos.resourcemanager.framework.role not working
> 
>
> Key: FLINK-7294
> URL: https://issues.apache.org/jira/browse/FLINK-7294
> Project: Flink
>  Issue Type: Bug
>  Components: Mesos
>Affects Versions: 1.3.1
>Reporter: Bhumika Bayani
>Priority: Critical
>
> I am using the above said setting in flink-conf.yaml
> e.g.
> mesos.resourcemanager.framework.role: mesos_role_tasks
> I see a flink-scheduler registered in mesos/frameworks tab with above said 
> role.
> But the scheduler fails to launch any tasks inspite of getting 
> resource-offers from mesos-agents with correct role.
> The error seen is:
> 2017-07-28 13:23:00,683 INFO  
> org.apache.flink.mesos.runtime.clusterframework.MesosFlinkResourceManager  - 
> Mesos task taskmanager-03768 failed, with a TaskManager in launch or 
> registration. State: TASK_ERROR Reason: REASON_TASK_INVALID (Task uses more 
> resources cpus(\*):1; mem(\*):1024; ports(\*):[4006-4007] than available 
> cpus(mesos_role_tasks):7.4; mem(mesos_role_tasks):45876; 
> ports(mesos_role_tasks):[4002-4129, 4131-4380, 4382-4809, 4811-4957, 
> 4959-4966, 4968-4979, 4981-5049, 31000-31196, 31198-31431, 31433-31607, 
> 31609-32000]; ephemeral_storage(mesos_role_tasks):37662; 
> efs_storage(mesos_role_tasks):8.79609e+12; disk(mesos_role_tasks):5115)
> The request is made for resources with * role. We do not have mesos running 
> anywhere with * role. Thus task manager never come up. 
> Am I missing any configuration?
> I am using flink version 1.3.1



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7326) Remove Flink's futures from RegisteredRpcConnection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107717#comment-16107717
 ] 

ASF GitHub Bot commented on FLINK-7326:
---

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4440

[FLINK-7326] [futures] Replace Flink's future with Java 8's 
CompletableFuture in RegisteredRpcConnection

## What is the purpose of the change

Replace Flink's future with Java 8's CompletableFuture in 
RegisteredRpcConnection.

This PR is based on #4429.

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)

## Documentation

  - Does this pull request introduce a new feature? (no)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink rfRegisteredRpcConnection

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4440.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4440


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit f20a4a083e545cb36d5ee11ccdff2f86244464f1
Author: Till Rohrmann 
Date:   2017-07-31T18:11:30Z

[FLINK-7326] [futures] Replace Flink's future with Java 8's 
CompletableFuture in RegisteredRpcConnection




> Remove Flink's futures from RegisteredRpcConnection
> ---
>
> Key: FLINK-7326
> URL: https://issues.apache.org/jira/browse/FLINK-7326
> Project: Flink
>  Issue Type: Sub-task
>  Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4440: [FLINK-7326] [futures] Replace Flink's future with...

[tillrohrmann]

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4440

[FLINK-7326] [futures] Replace Flink's future with Java 8's 
CompletableFuture in RegisteredRpcConnection

## What is the purpose of the change

Replace Flink's future with Java 8's CompletableFuture in 
RegisteredRpcConnection.

This PR is based on #4429.

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)

## Documentation

  - Does this pull request introduce a new feature? (no)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink rfRegisteredRpcConnection

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4440.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4440


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit f20a4a083e545cb36d5ee11ccdff2f86244464f1
Author: Till Rohrmann 
Date:   2017-07-31T18:11:30Z

[FLINK-7326] [futures] Replace Flink's future with Java 8's 
CompletableFuture in RegisteredRpcConnection




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (FLINK-7326) Remove Flink's futures from RegisteredRpcConnection

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-7326:


 Summary: Remove Flink's futures from RegisteredRpcConnection
 Key: FLINK-7326
 URL: https://issues.apache.org/jira/browse/FLINK-7326
 Project: Flink
  Issue Type: Sub-task
  Components: Distributed Coordination
Affects Versions: 1.4.0
Reporter: Till Rohrmann
Assignee: Till Rohrmann
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7325) Remove Flink's futures from MiniCluster

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107691#comment-16107691
 ] 

ASF GitHub Bot commented on FLINK-7325:
---

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4439

[FLINK-7325] [futures] Replace Flink's futures by Java 8's 
CompletableFuture in MiniCluster

## What is the purpose of the change

Replace Flink's futures by Java 8's CompletableFuture in MiniCluster.

This PR is based on #4429.

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)

## Documentation

  - Does this pull request introduce a new feature? (no)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink rfMiniCluster

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4439.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4439


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit e46423d86e7a72424ed7f8b4f4c4860414777471
Author: Till Rohrmann 
Date:   2017-07-31T17:55:50Z

[FLINK-7325] [futures] Replace Flink's futures by Java 8's 
CompletableFuture in MiniCluster




> Remove Flink's futures from MiniCluster
> ---
>
> Key: FLINK-7325
> URL: https://issues.apache.org/jira/browse/FLINK-7325
> Project: Flink
>  Issue Type: Sub-task
>  Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4439: [FLINK-7325] [futures] Replace Flink's futures by ...

[tillrohrmann]

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4439

[FLINK-7325] [futures] Replace Flink's futures by Java 8's 
CompletableFuture in MiniCluster

## What is the purpose of the change

Replace Flink's futures by Java 8's CompletableFuture in MiniCluster.

This PR is based on #4429.

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)

## Documentation

  - Does this pull request introduce a new feature? (no)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink rfMiniCluster

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4439.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4439


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit e46423d86e7a72424ed7f8b4f4c4860414777471
Author: Till Rohrmann 
Date:   2017-07-31T17:55:50Z

[FLINK-7325] [futures] Replace Flink's futures by Java 8's 
CompletableFuture in MiniCluster




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (FLINK-7325) Remove Flink's futures from MiniCluster

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-7325:


 Summary: Remove Flink's futures from MiniCluster
 Key: FLINK-7325
 URL: https://issues.apache.org/jira/browse/FLINK-7325
 Project: Flink
  Issue Type: Sub-task
  Components: Distributed Coordination
Affects Versions: 1.4.0
Reporter: Till Rohrmann
Assignee: Till Rohrmann
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7324) Remove Flink's futures from SlotPool

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107688#comment-16107688
 ] 

ASF GitHub Bot commented on FLINK-7324:
---

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4438

[FLINK-7324] [futures] Replace Flink's future with Java 8's 
CompletableFuture in SlotPool

## What is the purpose of the change

Replace Flink's future with Java 8's CompletableFuture in SlotPool.

This PR is based on #4429.

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes)

## Documentation

  - Does this pull request introduce a new feature? (no)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink rfSlotPool

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4438.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4438


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit c265fa386184b1fb940187c7f97c14e1542509de
Author: Till Rohrmann 
Date:   2017-07-31T17:35:14Z

[FLINK-7324] [futures] Replace Flink's future with Java 8's 
CompletableFuture in SlotPool




> Remove Flink's futures from SlotPool
> 
>
> Key: FLINK-7324
> URL: https://issues.apache.org/jira/browse/FLINK-7324
> Project: Flink
>  Issue Type: Sub-task
>  Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.4.0
>
>
> This only refers to the internal usage of Flink's {{Future}} usage in 
> {{SlotPool}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4438: [FLINK-7324] [futures] Replace Flink's future with...

[tillrohrmann]

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4438

[FLINK-7324] [futures] Replace Flink's future with Java 8's 
CompletableFuture in SlotPool

## What is the purpose of the change

Replace Flink's future with Java 8's CompletableFuture in SlotPool.

This PR is based on #4429.

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes)

## Documentation

  - Does this pull request introduce a new feature? (no)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink rfSlotPool

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4438.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4438


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit c265fa386184b1fb940187c7f97c14e1542509de
Author: Till Rohrmann 
Date:   2017-07-31T17:35:14Z

[FLINK-7324] [futures] Replace Flink's future with Java 8's 
CompletableFuture in SlotPool




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (FLINK-7324) Remove Flink's futures from SlotPool

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-7324:


 Summary: Remove Flink's futures from SlotPool
 Key: FLINK-7324
 URL: https://issues.apache.org/jira/browse/FLINK-7324
 Project: Flink
  Issue Type: Sub-task
  Components: Distributed Coordination
Affects Versions: 1.4.0
Reporter: Till Rohrmann
Assignee: Till Rohrmann
Priority: Minor


This only refers to the internal usage of Flink's {{Future}} usage in 
{{SlotPool}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7323) Remove Flink's futures from MasterHooks

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107619#comment-16107619
 ] 

ASF GitHub Bot commented on FLINK-7323:
---

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4437

[FLINK-7323] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in MasterHooks

## What is the purpose of the change

Replace Flink's futures with Java 8's CompletableFuture in MasterHooks.

This PR is based on #4429.

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes)

## Documentation

  - Does this pull request introduce a new feature? (yes)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink rfMasterHooks

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4437.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4437


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit 299cbb43392a0942b7ecd8712c8836e652f6d2ae
Author: Till Rohrmann 
Date:   2017-07-31T17:11:31Z

[FLINK-7323] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in MasterHooks




> Remove Flink's futures from MasterHooks
> ---
>
> Key: FLINK-7323
> URL: https://issues.apache.org/jira/browse/FLINK-7323
> Project: Flink
>  Issue Type: Sub-task
>  Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4437: [FLINK-7323] [futures] Replace Flink's futures wit...

[tillrohrmann]

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4437

[FLINK-7323] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in MasterHooks

## What is the purpose of the change

Replace Flink's futures with Java 8's CompletableFuture in MasterHooks.

This PR is based on #4429.

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes)

## Documentation

  - Does this pull request introduce a new feature? (yes)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink rfMasterHooks

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4437.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4437


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit 299cbb43392a0942b7ecd8712c8836e652f6d2ae
Author: Till Rohrmann 
Date:   2017-07-31T17:11:31Z

[FLINK-7323] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in MasterHooks




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (FLINK-7323) Remove Flink's futures from MasterHooks

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-7323:


 Summary: Remove Flink's futures from MasterHooks
 Key: FLINK-7323
 URL: https://issues.apache.org/jira/browse/FLINK-7323
 Project: Flink
  Issue Type: Sub-task
  Components: Distributed Coordination
Affects Versions: 1.4.0
Reporter: Till Rohrmann
Assignee: Till Rohrmann
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7322) Remove Flink's futures from CheckpointCoordinator

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107604#comment-16107604
 ] 

ASF GitHub Bot commented on FLINK-7322:
---

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4436

[FLINK-7322] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in CheckpointCoordinator

## What is the purpose of the change

Replace Flink's futures with Java 8's CompletableFuture in 
CheckpointCoordinator.

This PR is based on #4429.

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes)

## Documentation

  - Does this pull request introduce a new feature? (no)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink rfCheckpointCoordinator

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4436.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4436


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit 6ba44af6f8f2b0887c54020e3f5b090694ff0ff9
Author: Till Rohrmann 
Date:   2017-07-31T17:05:22Z

[FLINK-7322] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in CheckpointCoordinator




> Remove Flink's futures from CheckpointCoordinator
> -
>
> Key: FLINK-7322
> URL: https://issues.apache.org/jira/browse/FLINK-7322
> Project: Flink
>  Issue Type: Sub-task
>  Components: Distributed Coordination, State Backends, Checkpointing
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4436: [FLINK-7322] [futures] Replace Flink's futures wit...

[tillrohrmann]

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4436

[FLINK-7322] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in CheckpointCoordinator

## What is the purpose of the change

Replace Flink's futures with Java 8's CompletableFuture in 
CheckpointCoordinator.

This PR is based on #4429.

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes)

## Documentation

  - Does this pull request introduce a new feature? (no)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink rfCheckpointCoordinator

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4436.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4436


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit 6ba44af6f8f2b0887c54020e3f5b090694ff0ff9
Author: Till Rohrmann 
Date:   2017-07-31T17:05:22Z

[FLINK-7322] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in CheckpointCoordinator




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (FLINK-7322) Remove Flink's futures from CheckpointCoordinator

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-7322:


 Summary: Remove Flink's futures from CheckpointCoordinator
 Key: FLINK-7322
 URL: https://issues.apache.org/jira/browse/FLINK-7322
 Project: Flink
  Issue Type: Sub-task
  Components: Distributed Coordination, State Backends, Checkpointing
Affects Versions: 1.4.0
Reporter: Till Rohrmann
Assignee: Till Rohrmann
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7320) Remove Flink's futures from Scheduler

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107584#comment-16107584
 ] 

ASF GitHub Bot commented on FLINK-7320:
---

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4435

[FLINK-7320] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in Scheduler

## What is the purpose of the change

Replace Flink's futures with Java 8's CompletableFuture in Scheduler.

This PR is based on #4433.

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes)

## Documentation

  - Does this pull request introduce a new feature? (no)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink rfScheduler

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4435.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4435


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit 84dbf47a47e3f73bcf52db112efc36cb47f43180
Author: Till Rohrmann 
Date:   2017-07-31T15:55:06Z

[FLINK-7318] [futures] Replace Flink's futures in 
StackTraceSampleCoordinator with Java 8 CompletableFuture

commit 2fa9a9ebe5c669b6b9b20ec427d3faf10ffe5712
Author: Till Rohrmann 
Date:   2017-07-31T13:53:48Z

[FLINK-7314] [futures] Replace Flink's futures with CompletableFuture in 
TaskManagerLogHandler

commit f0344e4b554f440f966cd481ac245941d8a465ad
Author: Till Rohrmann 
Date:   2017-07-31T16:06:20Z

[FLINK-7319] [futures] Replace Flink's Futures with Java 8 
CompletableFuture in MesosResourceManager

commit d37fc6bd262d57324ee5f5b6e163b7450de9e286
Author: Till Rohrmann 
Date:   2017-07-31T14:33:21Z

[FLINK-7317] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in ExecutionGraph

Change FutureUtils.retry to work with CompletableFutures

Let ConjunctFutures extends CompletableFuture

Remove Flink's futures from ExecutionGraph

commit 62e01ca2cfd6f41fbdbd0640e52a81f92f3baeae
Author: Till Rohrmann 
Date:   2017-07-31T16:37:00Z

[FLINK-7320] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in Scheduler




> Remove Flink's futures from Scheduler
> -
>
> Key: FLINK-7320
> URL: https://issues.apache.org/jira/browse/FLINK-7320
> Project: Flink
>  Issue Type: Sub-task
>  Components: Scheduler
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4435: [FLINK-7320] [futures] Replace Flink's futures wit...

[tillrohrmann]

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4435

[FLINK-7320] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in Scheduler

## What is the purpose of the change

Replace Flink's futures with Java 8's CompletableFuture in Scheduler.

This PR is based on #4433.

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes)

## Documentation

  - Does this pull request introduce a new feature? (no)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink rfScheduler

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4435.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4435


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit 84dbf47a47e3f73bcf52db112efc36cb47f43180
Author: Till Rohrmann 
Date:   2017-07-31T15:55:06Z

[FLINK-7318] [futures] Replace Flink's futures in 
StackTraceSampleCoordinator with Java 8 CompletableFuture

commit 2fa9a9ebe5c669b6b9b20ec427d3faf10ffe5712
Author: Till Rohrmann 
Date:   2017-07-31T13:53:48Z

[FLINK-7314] [futures] Replace Flink's futures with CompletableFuture in 
TaskManagerLogHandler

commit f0344e4b554f440f966cd481ac245941d8a465ad
Author: Till Rohrmann 
Date:   2017-07-31T16:06:20Z

[FLINK-7319] [futures] Replace Flink's Futures with Java 8 
CompletableFuture in MesosResourceManager

commit d37fc6bd262d57324ee5f5b6e163b7450de9e286
Author: Till Rohrmann 
Date:   2017-07-31T14:33:21Z

[FLINK-7317] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in ExecutionGraph

Change FutureUtils.retry to work with CompletableFutures

Let ConjunctFutures extends CompletableFuture

Remove Flink's futures from ExecutionGraph

commit 62e01ca2cfd6f41fbdbd0640e52a81f92f3baeae
Author: Till Rohrmann 
Date:   2017-07-31T16:37:00Z

[FLINK-7320] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in Scheduler




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLINK-7321) Remove Flink's futures from HeartbeatManager

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107576#comment-16107576
 ] 

ASF GitHub Bot commented on FLINK-7321:
---

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4434

[FLINK-7321] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in HeartbeatManager

## What is the purpose of the change

Replace Flink's futures with Java 8's CompletableFuture in HeartbeatManager.

This PR is based on #4429.

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)

## Documentation

  - Does this pull request introduce a new feature? (no)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink rfHearbeatManager

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4434.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4434


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit 8425e85f3ea0499fb59fc3c25c035a622e3b282d
Author: Till Rohrmann 
Date:   2017-07-31T16:47:22Z

[FLINK-7321] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in HeartbeatManager




> Remove Flink's futures from HeartbeatManager
> 
>
> Key: FLINK-7321
> URL: https://issues.apache.org/jira/browse/FLINK-7321
> Project: Flink
>  Issue Type: Sub-task
>  Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4434: [FLINK-7321] [futures] Replace Flink's futures wit...

[tillrohrmann]

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4434

[FLINK-7321] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in HeartbeatManager

## What is the purpose of the change

Replace Flink's futures with Java 8's CompletableFuture in HeartbeatManager.

This PR is based on #4429.

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)

## Documentation

  - Does this pull request introduce a new feature? (no)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink rfHearbeatManager

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4434.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4434


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit 8425e85f3ea0499fb59fc3c25c035a622e3b282d
Author: Till Rohrmann 
Date:   2017-07-31T16:47:22Z

[FLINK-7321] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in HeartbeatManager




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (FLINK-7321) Remove Flink's futures from HeartbeatManager

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-7321:


 Summary: Remove Flink's futures from HeartbeatManager
 Key: FLINK-7321
 URL: https://issues.apache.org/jira/browse/FLINK-7321
 Project: Flink
  Issue Type: Sub-task
  Components: Distributed Coordination
Affects Versions: 1.4.0
Reporter: Till Rohrmann
Assignee: Till Rohrmann
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (FLINK-7320) Remove Flink's futures from Scheduler

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-7320:


 Summary: Remove Flink's futures from Scheduler
 Key: FLINK-7320
 URL: https://issues.apache.org/jira/browse/FLINK-7320
 Project: Flink
  Issue Type: Sub-task
  Components: Scheduler
Affects Versions: 1.4.0
Reporter: Till Rohrmann
Assignee: Till Rohrmann
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Closed] (FLINK-6732) Activate strict checkstyle for flink-java

[Chesnay Schepler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLINK-6732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chesnay Schepler closed FLINK-6732.
---
   Resolution: Fixed
Fix Version/s: 1.4.0

1.4: 3c42557f3083182b0ab66d15cb6ec8452b59464c

> Activate strict checkstyle for flink-java
> -
>
> Key: FLINK-6732
> URL: https://issues.apache.org/jira/browse/FLINK-6732
> Project: Flink
>  Issue Type: Sub-task
>  Components: Java API
>Reporter: Chesnay Schepler
>Assignee: Dawid Wysakowicz
> Fix For: 1.4.0
>
>
> Long term issue for incrementally introducing the strict checkstyle to 
> flink-java.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Closed] (FLINK-7185) Activate checkstyle flink-java/io

[Chesnay Schepler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLINK-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chesnay Schepler closed FLINK-7185.
---
   Resolution: Fixed
Fix Version/s: 1.4.0

1.4: 0c9c9fb5cb7a8a27d444db5c725c8abd792ca761

> Activate checkstyle flink-java/io
> -
>
> Key: FLINK-7185
> URL: https://issues.apache.org/jira/browse/FLINK-7185
> Project: Flink
>  Issue Type: Improvement
>  Components: Build System
>Reporter: Dawid Wysakowicz
>Assignee: Dawid Wysakowicz
>Priority: Trivial
> Fix For: 1.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Closed] (FLINK-7191) Activate checkstyle flink-java/operators/translation

[Chesnay Schepler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLINK-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chesnay Schepler closed FLINK-7191.
---
   Resolution: Fixed
Fix Version/s: 1.4.0

1.4: 8e975362312c727fd602429778bc1c3628b95619

> Activate checkstyle flink-java/operators/translation
> 
>
> Key: FLINK-7191
> URL: https://issues.apache.org/jira/browse/FLINK-7191
> Project: Flink
>  Issue Type: Improvement
>  Components: Build System
>Reporter: Dawid Wysakowicz
>Assignee: Dawid Wysakowicz
>Priority: Trivial
> Fix For: 1.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7191) Activate checkstyle flink-java/operators/translation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107531#comment-16107531
 ] 

ASF GitHub Bot commented on FLINK-7191:
---

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/4334


> Activate checkstyle flink-java/operators/translation
> 
>
> Key: FLINK-7191
> URL: https://issues.apache.org/jira/browse/FLINK-7191
> Project: Flink
>  Issue Type: Improvement
>  Components: Build System
>Reporter: Dawid Wysakowicz
>Assignee: Dawid Wysakowicz
>Priority: Trivial
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7185) Activate checkstyle flink-java/io

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107529#comment-16107529
 ] 

ASF GitHub Bot commented on FLINK-7185:
---

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/4340


> Activate checkstyle flink-java/io
> -
>
> Key: FLINK-7185
> URL: https://issues.apache.org/jira/browse/FLINK-7185
> Project: Flink
>  Issue Type: Improvement
>  Components: Build System
>Reporter: Dawid Wysakowicz
>Assignee: Dawid Wysakowicz
>Priority: Trivial
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4334: [FLINK-7191] Activate checkstyle flink-java/operat...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/4334


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #4340: [FLINK-7185] Activate checkstyle flink-java/io

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/4340


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLINK-7317) Remove Flink's futures from ExecutionGraph

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107527#comment-16107527
 ] 

ASF GitHub Bot commented on FLINK-7317:
---

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4433

[FLINK-7317] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in ExecutionGraph

## What is the purpose of the change

Replace Flink's `Futures` with Java 8's `CompletableFuture` in 
`ExecutionGraph`

This PR is based #4429, #4431, #4432, #4430.

## Brief change log

*(for example:)*
  - Use `CompletableFuture` in `ExecutionGraph`
  - Change `FutureUtils#retry` to work with `CompletableFuture`
  - Let `ConjunctFuture` extend from `CompletableFuture`

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes)

## Documentation

  - Does this pull request introduce a new feature? (no)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink rfScheduling

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4433.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4433


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit 84dbf47a47e3f73bcf52db112efc36cb47f43180
Author: Till Rohrmann 
Date:   2017-07-31T15:55:06Z

[FLINK-7318] [futures] Replace Flink's futures in 
StackTraceSampleCoordinator with Java 8 CompletableFuture

commit 2fa9a9ebe5c669b6b9b20ec427d3faf10ffe5712
Author: Till Rohrmann 
Date:   2017-07-31T13:53:48Z

[FLINK-7314] [futures] Replace Flink's futures with CompletableFuture in 
TaskManagerLogHandler

commit f0344e4b554f440f966cd481ac245941d8a465ad
Author: Till Rohrmann 
Date:   2017-07-31T16:06:20Z

[FLINK-7319] [futures] Replace Flink's Futures with Java 8 
CompletableFuture in MesosResourceManager

commit d37fc6bd262d57324ee5f5b6e163b7450de9e286
Author: Till Rohrmann 
Date:   2017-07-31T14:33:21Z

[FLINK-7317] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in ExecutionGraph

Change FutureUtils.retry to work with CompletableFutures

Let ConjunctFutures extends CompletableFuture

Remove Flink's futures from ExecutionGraph




> Remove Flink's futures from ExecutionGraph
> --
>
> Key: FLINK-7317
> URL: https://issues.apache.org/jira/browse/FLINK-7317
> Project: Flink
>  Issue Type: Sub-task
>  Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
> Fix For: 1.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4433: [FLINK-7317] [futures] Replace Flink's futures wit...

[tillrohrmann]

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4433

[FLINK-7317] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in ExecutionGraph

## What is the purpose of the change

Replace Flink's `Futures` with Java 8's `CompletableFuture` in 
`ExecutionGraph`

This PR is based #4429, #4431, #4432, #4430.

## Brief change log

*(for example:)*
  - Use `CompletableFuture` in `ExecutionGraph`
  - Change `FutureUtils#retry` to work with `CompletableFuture`
  - Let `ConjunctFuture` extend from `CompletableFuture`

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes)

## Documentation

  - Does this pull request introduce a new feature? (no)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink rfScheduling

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4433.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4433


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit 84dbf47a47e3f73bcf52db112efc36cb47f43180
Author: Till Rohrmann 
Date:   2017-07-31T15:55:06Z

[FLINK-7318] [futures] Replace Flink's futures in 
StackTraceSampleCoordinator with Java 8 CompletableFuture

commit 2fa9a9ebe5c669b6b9b20ec427d3faf10ffe5712
Author: Till Rohrmann 
Date:   2017-07-31T13:53:48Z

[FLINK-7314] [futures] Replace Flink's futures with CompletableFuture in 
TaskManagerLogHandler

commit f0344e4b554f440f966cd481ac245941d8a465ad
Author: Till Rohrmann 
Date:   2017-07-31T16:06:20Z

[FLINK-7319] [futures] Replace Flink's Futures with Java 8 
CompletableFuture in MesosResourceManager

commit d37fc6bd262d57324ee5f5b6e163b7450de9e286
Author: Till Rohrmann 
Date:   2017-07-31T14:33:21Z

[FLINK-7317] [futures] Replace Flink's futures with Java 8's 
CompletableFuture in ExecutionGraph

Change FutureUtils.retry to work with CompletableFutures

Let ConjunctFutures extends CompletableFuture

Remove Flink's futures from ExecutionGraph




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLINK-7318) Remove Flink's futures from StackTraceSampleCoordinator

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107515#comment-16107515
 ] 

ASF GitHub Bot commented on FLINK-7318:
---

Github user tillrohrmann commented on a diff in the pull request:

https://github.com/apache/flink/pull/4431#discussion_r130397834
  
--- Diff: 
flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/BackPressureStatsTracker.java
 ---
@@ -177,7 +177,7 @@ public boolean 
triggerStackTraceSample(ExecutionJobVertex vertex) {
LOG.debug("Triggering stack 
trace sample for tasks: " + Arrays.toString(vertex.getTaskVertices()));
}
 
-   Future sample = 
coordinator.triggerStackTraceSample(
+   CompletableFuture 
sample = coordinator.triggerStackTraceSample(
--- End diff --

The Java `Future` does not allow you to apply transformations to it. That 
was only introduced with Java 8's `CompletableFuture`.


> Remove Flink's futures from StackTraceSampleCoordinator
> ---
>
> Key: FLINK-7318
> URL: https://issues.apache.org/jira/browse/FLINK-7318
> Project: Flink
>  Issue Type: Sub-task
>  Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4431: [FLINK-7318] [futures] Replace Flink's futures in ...

[tillrohrmann]

Github user tillrohrmann commented on a diff in the pull request:

https://github.com/apache/flink/pull/4431#discussion_r130397834
  
--- Diff: 
flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/BackPressureStatsTracker.java
 ---
@@ -177,7 +177,7 @@ public boolean 
triggerStackTraceSample(ExecutionJobVertex vertex) {
LOG.debug("Triggering stack 
trace sample for tasks: " + Arrays.toString(vertex.getTaskVertices()));
}
 
-   Future sample = 
coordinator.triggerStackTraceSample(
+   CompletableFuture 
sample = coordinator.triggerStackTraceSample(
--- End diff --

The Java `Future` does not allow you to apply transformations to it. That 
was only introduced with Java 8's `CompletableFuture`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLINK-7318) Remove Flink's futures from StackTraceSampleCoordinator

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107511#comment-16107511
 ] 

ASF GitHub Bot commented on FLINK-7318:
---

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4431#discussion_r130396606
  
--- Diff: 
flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/BackPressureStatsTracker.java
 ---
@@ -177,7 +177,7 @@ public boolean 
triggerStackTraceSample(ExecutionJobVertex vertex) {
LOG.debug("Triggering stack 
trace sample for tasks: " + Arrays.toString(vertex.getTaskVertices()));
}
 
-   Future sample = 
coordinator.triggerStackTraceSample(
+   CompletableFuture 
sample = coordinator.triggerStackTraceSample(
--- End diff --

Why not you not returning a plain `Future` here? Shouldn't we only 
explicitly return a `CompletableFuture` if we expect the caller to complete it?


> Remove Flink's futures from StackTraceSampleCoordinator
> ---
>
> Key: FLINK-7318
> URL: https://issues.apache.org/jira/browse/FLINK-7318
> Project: Flink
>  Issue Type: Sub-task
>  Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4431: [FLINK-7318] [futures] Replace Flink's futures in ...

[zentol]

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4431#discussion_r130396606
  
--- Diff: 
flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/BackPressureStatsTracker.java
 ---
@@ -177,7 +177,7 @@ public boolean 
triggerStackTraceSample(ExecutionJobVertex vertex) {
LOG.debug("Triggering stack 
trace sample for tasks: " + Arrays.toString(vertex.getTaskVertices()));
}
 
-   Future sample = 
coordinator.triggerStackTraceSample(
+   CompletableFuture 
sample = coordinator.triggerStackTraceSample(
--- End diff --

Why not you not returning a plain `Future` here? Shouldn't we only 
explicitly return a `CompletableFuture` if we expect the caller to complete it?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLINK-7319) Remove Flink's futures from MesosResourceManager

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107507#comment-16107507
 ] 

ASF GitHub Bot commented on FLINK-7319:
---

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4432

[FLINK-7319] [futures] Replace Flink's Futures with Java 8 
CompletableFuture in MesosResourceManager

## What is the purpose of the change

Replace Flink's `Futures` with Java 8 `CompletableFuture` in 
`MesosResourceManager` (only the internal use of it).

This PR is based on #4429.

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)

## Documentation

  - Does this pull request introduce a new feature? (no)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink rfMesosResourceManager

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4432.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4432


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit 9c3cc4e37d2c67551ffb21ad6f7dcaa81e0d4d50
Author: Till Rohrmann 
Date:   2017-07-31T16:06:20Z

[FLINK-7319] [futures] Replace Flink's Futures with Java 8 
CompletableFuture in MesosResourceManager




> Remove Flink's futures from MesosResourceManager
> 
>
> Key: FLINK-7319
> URL: https://issues.apache.org/jira/browse/FLINK-7319
> Project: Flink
>  Issue Type: Sub-task
>  Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.4.0
>
>
> Only the internal use of Flink's {{Futures}} first. The interface will be 
> changed once we change the RPC abstraction to use Java 8 
> {{CompletableFutures}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4432: [FLINK-7319] [futures] Replace Flink's Futures wit...

[tillrohrmann]

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4432

[FLINK-7319] [futures] Replace Flink's Futures with Java 8 
CompletableFuture in MesosResourceManager

## What is the purpose of the change

Replace Flink's `Futures` with Java 8 `CompletableFuture` in 
`MesosResourceManager` (only the internal use of it).

This PR is based on #4429.

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)

## Documentation

  - Does this pull request introduce a new feature? (no)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink rfMesosResourceManager

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4432.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4432


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit 9c3cc4e37d2c67551ffb21ad6f7dcaa81e0d4d50
Author: Till Rohrmann 
Date:   2017-07-31T16:06:20Z

[FLINK-7319] [futures] Replace Flink's Futures with Java 8 
CompletableFuture in MesosResourceManager




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (FLINK-7319) Remove Flink's futures from MesosResourceManager

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-7319:


 Summary: Remove Flink's futures from MesosResourceManager
 Key: FLINK-7319
 URL: https://issues.apache.org/jira/browse/FLINK-7319
 Project: Flink
  Issue Type: Sub-task
  Components: Distributed Coordination
Affects Versions: 1.4.0
Reporter: Till Rohrmann
Assignee: Till Rohrmann
Priority: Minor


Only the internal use of Flink's {{Futures}} first. The interface will be 
changed once we change the RPC abstraction to use Java 8 {{CompletableFutures}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7318) Remove Flink's futures from StackTraceSampleCoordinator

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107496#comment-16107496
 ] 

ASF GitHub Bot commented on FLINK-7318:
---

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4431

[FLINK-7318] [futures] Replace Flink's futures in 
StackTraceSampleCoordinator with Java 8 CompletableFuture

## What is the purpose of the change

Replace Flink's futures in `StackTraceSampleCoordinator` and 
`BackPressureStatsTracker` with Java 8 `CompletableFuture`.

This PR is based on #4429.

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (yes / no / don't know)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)

## Documentation

  - Does this pull request introduce a new feature? (no)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink 
rfStackTraceSampleCoordinator

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4431.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4431


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit 84dbf47a47e3f73bcf52db112efc36cb47f43180
Author: Till Rohrmann 
Date:   2017-07-31T15:55:06Z

[FLINK-7318] [futures] Replace Flink's futures in 
StackTraceSampleCoordinator with Java 8 CompletableFuture




> Remove Flink's futures from StackTraceSampleCoordinator
> ---
>
> Key: FLINK-7318
> URL: https://issues.apache.org/jira/browse/FLINK-7318
> Project: Flink
>  Issue Type: Sub-task
>  Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink pull request #4431: [FLINK-7318] [futures] Replace Flink's futures in ...

[tillrohrmann]

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4431

[FLINK-7318] [futures] Replace Flink's futures in 
StackTraceSampleCoordinator with Java 8 CompletableFuture

## What is the purpose of the change

Replace Flink's futures in `StackTraceSampleCoordinator` and 
`BackPressureStatsTracker` with Java 8 `CompletableFuture`.

This PR is based on #4429.

## Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (yes / no / don't know)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)

## Documentation

  - Does this pull request introduce a new feature? (no)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink 
rfStackTraceSampleCoordinator

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4431.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4431


commit afe1d171132bb3724e672a1c4ce74a3f7c185908
Author: Till Rohrmann 
Date:   2017-07-31T13:07:18Z

[FLINK-7313] [futures] Add Flink future and Scala future to Java 8 
CompletableFuture conversion

Add DirectExecutionContext

Add Scala Future to Java 8 CompletableFuture utility to FutureUtils

Add Flink future to Java 8's CompletableFuture conversion utility to 
FutureUtils

Add base class for Flink's unchecked future exceptions

commit 84dbf47a47e3f73bcf52db112efc36cb47f43180
Author: Till Rohrmann 
Date:   2017-07-31T15:55:06Z

[FLINK-7318] [futures] Replace Flink's futures in 
StackTraceSampleCoordinator with Java 8 CompletableFuture




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (FLINK-7318) Remove Flink's futures from StackTraceSampleCoordinator

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-7318:


 Summary: Remove Flink's futures from StackTraceSampleCoordinator
 Key: FLINK-7318
 URL: https://issues.apache.org/jira/browse/FLINK-7318
 Project: Flink
  Issue Type: Sub-task
  Components: Distributed Coordination
Affects Versions: 1.4.0
Reporter: Till Rohrmann
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (FLINK-7318) Remove Flink's futures from StackTraceSampleCoordinator

[Till Rohrmann (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLINK-7318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Till Rohrmann reassigned FLINK-7318:


Assignee: Till Rohrmann

> Remove Flink's futures from StackTraceSampleCoordinator
> ---
>
> Key: FLINK-7318
> URL: https://issues.apache.org/jira/browse/FLINK-7318
> Project: Flink
>  Issue Type: Sub-task
>  Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (FLINK-7317) Remove Flink's futures from ExecutionGraph

[Till Rohrmann (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLINK-7317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Till Rohrmann reassigned FLINK-7317:


Assignee: Till Rohrmann

> Remove Flink's futures from ExecutionGraph
> --
>
> Key: FLINK-7317
> URL: https://issues.apache.org/jira/browse/FLINK-7317
> Project: Flink
>  Issue Type: Sub-task
>  Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
> Fix For: 1.4.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (FLINK-7317) Remove Flink's futures from ExecutionGraph

[Till Rohrmann (JIRA)]

Till Rohrmann created FLINK-7317:


 Summary: Remove Flink's futures from ExecutionGraph
 Key: FLINK-7317
 URL: https://issues.apache.org/jira/browse/FLINK-7317
 Project: Flink
  Issue Type: Sub-task
  Components: Distributed Coordination
Affects Versions: 1.4.0
Reporter: Till Rohrmann






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink issue #4334: [FLINK-7191] Activate checkstyle flink-java/operators/tra...

[zentol]

Github user zentol commented on the issue:

https://github.com/apache/flink/pull/4334
  
Will address the comment while merging.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLINK-7191) Activate checkstyle flink-java/operators/translation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107379#comment-16107379
 ] 

ASF GitHub Bot commented on FLINK-7191:
---

Github user zentol commented on the issue:

https://github.com/apache/flink/pull/4334
  
Will address the comment while merging.


> Activate checkstyle flink-java/operators/translation
> 
>
> Key: FLINK-7191
> URL: https://issues.apache.org/jira/browse/FLINK-7191
> Project: Flink
>  Issue Type: Improvement
>  Components: Build System
>Reporter: Dawid Wysakowicz
>Assignee: Dawid Wysakowicz
>Priority: Trivial
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7185) Activate checkstyle flink-java/io

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107367#comment-16107367
 ] 

ASF GitHub Bot commented on FLINK-7185:
---

Github user zentol commented on the issue:

https://github.com/apache/flink/pull/4340
  
merging.


> Activate checkstyle flink-java/io
> -
>
> Key: FLINK-7185
> URL: https://issues.apache.org/jira/browse/FLINK-7185
> Project: Flink
>  Issue Type: Improvement
>  Components: Build System
>Reporter: Dawid Wysakowicz
>Assignee: Dawid Wysakowicz
>Priority: Trivial
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] flink issue #4340: [FLINK-7185] Activate checkstyle flink-java/io

[zentol]

Github user zentol commented on the issue:

https://github.com/apache/flink/pull/4340
  
merging.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (FLINK-7316) always use off-heap network buffers

[Nico Kruber (JIRA)]

Nico Kruber created FLINK-7316:
--

 Summary: always use off-heap network buffers
 Key: FLINK-7316
 URL: https://issues.apache.org/jira/browse/FLINK-7316
 Project: Flink
  Issue Type: Sub-task
  Components: Core, Network
Affects Versions: 1.4.0
Reporter: Nico Kruber
Assignee: Nico Kruber


In order to send flink buffers through netty into the network, we need to make 
the buffers use off-heap memory. Otherwise, there will be a hidden copy 
happening in the NIO stack.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (FLINK-7310) always use HybridMemorySegment

[Nico Kruber (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLINK-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nico Kruber updated FLINK-7310:
---
Issue Type: Sub-task  (was: Improvement)
Parent: FLINK-7315

> always use HybridMemorySegment
> --
>
> Key: FLINK-7310
> URL: https://issues.apache.org/jira/browse/FLINK-7310
> Project: Flink
>  Issue Type: Sub-task
>  Components: Core
>Affects Versions: 1.4.0
>Reporter: Nico Kruber
>Assignee: Nico Kruber
>
> For future changes to the network buffers (sending our own off-heap buffers 
> through to netty), we cannot use {{HeapMemorySegment}} anymore and need to 
> rely on {{HybridMemorySegment}} instead.
> We should thus drop any code that loads the {{HeapMemorySegment}} (it is 
> still available if needed) in favour of the {{HybridMemorySegment}} which is 
> able to work on both heap and off-heap memory.
> FYI: For the performance penalty of this change compared to using 
> {{HeapMemorySegment}} alone, see this interesting blob article (from 2015):
> https://flink.apache.org/news/2015/09/16/off-heap-memory.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (FLINK-7315) use flink's buffers in netty

[Nico Kruber (JIRA)]

Nico Kruber created FLINK-7315:
--

 Summary: use flink's buffers in netty
 Key: FLINK-7315
 URL: https://issues.apache.org/jira/browse/FLINK-7315
 Project: Flink
  Issue Type: Improvement
  Components: Core, Network
Affects Versions: 1.4.0
Reporter: Nico Kruber
Assignee: Nico Kruber


The goal of this change is to avoid the step in the channel encoder and decoder 
pipelines where flink buffers are copied into netty buffers. Instead, netty 
should directly send flink buffers to the network.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Closed] (FLINK-7283) PythonPlanBinderTest issues with python paths

[Chesnay Schepler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLINK-7283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chesnay Schepler closed FLINK-7283.
---
Resolution: Fixed

1.3: 2455df962c76d20f1c07a57d6ed0118d1d1a067c

> PythonPlanBinderTest issues with python paths
> -
>
> Key: FLINK-7283
> URL: https://issues.apache.org/jira/browse/FLINK-7283
> Project: Flink
>  Issue Type: Bug
>  Components: Python API, Tests
>Affects Versions: 1.3.0, 1.3.1, 1.4.0, 1.3.2
>Reporter: Nico Kruber
>Assignee: Nico Kruber
> Fix For: 1.4.0, 1.3.3
>
>
> There are some issues with {{PythonPlanBinderTest}} and the Python2/3 tests:
> - the path is not set correctly (only inside {{config}}, not the 
> {{configuration}} that is passed on to the {{PythonPlanBinder}}
> - linux distributions have become quite inventive regarding python binary 
> names: some offer {{python}} as Python 2, some as Python 3. Similarly, 
> {{python3}} and/or {{python2}} may not be available. If we really want to 
> test both, we need to take this into account.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)