[
https://issues.apache.org/jira/browse/BEAM-9541?focusedWorklogId=530592&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-530592
]
ASF GitHub Bot logged work on BEAM-9541:
----------------------------------------
Author: ASF GitHub Bot
Created on: 04/Jan/21 10:31
Start Date: 04/Jan/21 10:31
Worklog Time Spent: 10m
Work Description: mxm commented on a change in pull request #13633:
URL: https://github.com/apache/beam/pull/13633#discussion_r551233404
##########
File path:
runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkExecutionEnvironments.java
##########
@@ -375,4 +377,10 @@ private static void setManagedMemoryByFraction(final
Configuration config) {
config.setString("taskmanager.memory.managed.size",
String.valueOf(managedMemorySize));
}
}
+
+ private static void disableClassLoaderLeakCheck(final Configuration config) {
Review comment:
In any case, we should add a note or JIRA ticket to why this is
necessary.
##########
File path: runners/flink/flink_runner.gradle
##########
@@ -108,17 +108,14 @@ test {
if (System.getProperty("beamSurefireArgline")) {
jvmArgs System.getProperty("beamSurefireArgline")
}
- // TODO Running tests of all Flink versions in parallel can be too harsh on
Jenkins memory
- // Run them serially for now, to avoid "Exit code 137", i.e. Jenkins host
killing the Gradle test process
- if (project.path == ":runners:flink:1.9") {
- mustRunAfter(":runners:flink:1.8:test")
- } else if (project.path == ":runners:flink:1.10") {
- mustRunAfter(":runners:flink:1.8:test")
- mustRunAfter(":runners:flink:1.9:test")
- } else if (project.path == ":runners:flink:1.11") {
- mustRunAfter(":runners:flink:1.8:test")
- mustRunAfter(":runners:flink:1.9:test")
- mustRunAfter(":runners:flink:1.10:test")
+ // TODO(BEAM-6418) Running tests of all Flink versions in parallel can be
too harsh on Jenkins memory.
+ // Run them serially for now, to avoid "Exit code 137", i.e. Jenkins host
killing the Gradle test process.
+ def flink_minor_version = project.path.split(':').last()
+ for (version in project.ext.allFlinkVersions) {
+ if (version == flink_minor_version) {
+ break
+ }
+ mustRunAfter(":runners:flink:${version}:test")
Review comment:
Personal preference but I find this easier to read:
```suggestion
if (version != flink_minor_version) {
mustRunAfter(":runners:flink:${version}:test")
}
```
##########
File path:
runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkExecutionEnvironments.java
##########
@@ -375,4 +377,10 @@ private static void setManagedMemoryByFraction(final
Configuration config) {
config.setString("taskmanager.memory.managed.size",
String.valueOf(managedMemorySize));
}
}
+
+ private static void disableClassLoaderLeakCheck(final Configuration config) {
Review comment:
I believe this is because of GRPC which may attempt to load classes
after the Flink job has been removed. Loading new classes on closed
classloaders will error without this setting.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 530592)
Time Spent: 4h 10m (was: 4h)
> Single source of truth for supported Flink versions
> ---------------------------------------------------
>
> Key: BEAM-9541
> URL: https://issues.apache.org/jira/browse/BEAM-9541
> Project: Beam
> Issue Type: Improvement
> Components: runner-flink
> Reporter: Kyle Weaver
> Assignee: Kyle Weaver
> Priority: P2
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> Currently, there is a large number of hard-coded references to the supported
> Flink versions (either the newest version, or a list of all supported
> versions). These should be condensed to a single source of truth to make
> upgrades easier and more robust.
> Previously, we awked the list of Flink subdirectories, but this is not
> reliable because old, unsupported Flink versions can leave build directories
> behind. It would be better to read versions from a handwritten file.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
