[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16443687#comment-16443687 ] ASF GitHub Bot commented on FLINK-8955: --- Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/5780 > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16442596#comment-16442596 ] ASF GitHub Bot commented on FLINK-8955: --- Github user zentol commented on the issue: https://github.com/apache/flink/pull/5780 merging > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441190#comment-16441190 ] ASF GitHub Bot commented on FLINK-8955: --- Github user zentol commented on the issue: https://github.com/apache/flink/pull/5780 checkstyle > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441019#comment-16441019 ] ASF GitHub Bot commented on FLINK-8955: --- Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/5780 still failures because of this? > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440768#comment-16440768 ] ASF GitHub Bot commented on FLINK-8955: --- Github user zentol commented on the issue: https://github.com/apache/flink/pull/5780 I may have found the case, we have to set `MANAGED_MEMORY_SIZE` for this test. > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440691#comment-16440691 ] ASF GitHub Bot commented on FLINK-8955: --- Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/5780 Failing reproducibly, and only on this branch? > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16439920#comment-16439920 ] ASF GitHub Bot commented on FLINK-8955: --- Github user zentol commented on the issue: https://github.com/apache/flink/pull/5780 travis is still failing with mysterious error messages.. ``` 15:58:07.061 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (integration-tests) on project flink-tests_2.11: ExecutionException: java.lang.RuntimeException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called? 15:58:07.061 [ERROR] Command was /bin/sh -c cd /home/travis/build/apache/flink/flink-tests/target && /usr/lib/jvm/java-8-oracle/jre/bin/java -Xms256m -Xmx2048m -Dmvn.forkNumber=1 -XX:+UseG1GC -jar /home/travis/build/apache/flink/flink-tests/target/surefire/surefirebooter5397929458643043612.jar /home/travis/build/apache/flink/flink-tests/target/surefire/surefire4818404066082555175tmp /home/travis/build/apache/flink/flink-tests/target/surefire/surefire_1027420331622776884606tmp ``` > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[
https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16439536#comment-16439536
]
ASF GitHub Bot commented on FLINK-8955:
---
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181759432
--- Diff:
flink-tests/src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java
---
@@ -202,15 +204,27 @@ public void
testCheckpointedStreamingClassloaderJobWithCustomClassLoader() throw
Collections.singleton(new
Path(STREAMING_CHECKPOINTED_PROG_JAR_FILE)),
Collections.emptyList());
- // Program should terminate with a 'SuccessException':
- // we can not access the SuccessException here when executing
the tests with maven, because its not available in the jar.
- expectedException.expectCause(
- Matchers.hasProperty("cause",
- hasProperty("class",
- hasProperty("canonicalName", equalTo(
-
"org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException");
-
- streamingCheckpointedProg.invokeInteractiveModeForExecution();
+ try {
--- End diff --
my guess is that the exception stack has changed. The SuccessException is
no longer the direct cause of the top-level exception.
```
Expected: exception with cause hasProperty("cause", hasProperty("class",
hasProperty("canonicalName",
"org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException")))
but: cause property 'cause' property 'class' null
Stacktrace was: org.apache.flink.client.program.ProgramInvocationException:
The main method caused an error.
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:545)
at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:420)
at
org.apache.flink.test.classloading.ClassLoaderITCase.testCheckpointedStreamingClassloaderJobWithCustomClassLoader(ClassLoaderITCase.java:212)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at
org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:239)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[
https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16439488#comment-16439488
]
ASF GitHub Bot commented on FLINK-8955:
---
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181746661
--- Diff:
flink-tests/src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java
---
@@ -202,15 +204,27 @@ public void
testCheckpointedStreamingClassloaderJobWithCustomClassLoader() throw
Collections.singleton(new
Path(STREAMING_CHECKPOINTED_PROG_JAR_FILE)),
Collections.emptyList());
- // Program should terminate with a 'SuccessException':
- // we can not access the SuccessException here when executing
the tests with maven, because its not available in the jar.
- expectedException.expectCause(
- Matchers.hasProperty("cause",
- hasProperty("class",
- hasProperty("canonicalName", equalTo(
-
"org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException");
-
- streamingCheckpointedProg.invokeInteractiveModeForExecution();
+ try {
--- End diff --
Why doesn't the old code work here anymore?
> Port ClassLoaderITCase to flip6
> ---
>
> Key: FLINK-8955
> URL: https://issues.apache.org/jira/browse/FLINK-8955
> Project: Flink
> Issue Type: Improvement
> Components: Tests
>Affects Versions: 1.5.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Blocker
> Fix For: 1.5.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438702#comment-16438702 ] ASF GitHub Bot commented on FLINK-8955: --- Github user zentol commented on the issue: https://github.com/apache/flink/pull/5780 I've cut down the PR to only not make any changes outside the tests, fixed the checkstyle violations and the failing test. > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[
https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438700#comment-16438700
]
ASF GitHub Bot commented on FLINK-8955:
---
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181582943
--- Diff:
flink-tests/src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java
---
@@ -196,21 +188,23 @@ public void
testCheckpointedStreamingClassloaderJobWithCustomClassLoader() throw
// the test also ensures that user specific exceptions are
serializable between JobManager <--> JobClient.
PackagedProgram streamingCheckpointedProg = new
PackagedProgram(new File(STREAMING_CHECKPOINTED_PROG_JAR_FILE));
- TestStreamEnvironment.setAsContext(
- testCluster,
+ TestEnvironment.setAsContext(
+ MINI_CLUSTER_RESOURCE,
parallelism,
Collections.singleton(new
Path(STREAMING_CHECKPOINTED_PROG_JAR_FILE)),
Collections.emptyList());
- // Program should terminate with a 'SuccessException':
- // we can not access the SuccessException here when executing
the tests with maven, because its not available in the jar.
- expectedException.expectCause(
- Matchers.hasProperty("cause",
- hasProperty("class",
- hasProperty("canonicalName", equalTo(
-
"org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException");
-
- streamingCheckpointedProg.invokeInteractiveModeForExecution();
+ try {
+
streamingCheckpointedProg.invokeInteractiveModeForExecution();
+ } catch (Exception e) {
+ // Program should terminate with a 'SuccessException':
+ // we can not access the SuccessException here when
executing the tests with maven, because its not available in the jar.
+ Optional expectedCause =
ExceptionUtils.findThrowable(e,
+ candidate ->
candidate.getClass().getCanonicalName().equals("org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException"));
--- End diff --
The test failed since the `TestEnvironment` was set as the context, and not
the `TestStreamEnvironment` as intended.
> Port ClassLoaderITCase to flip6
> ---
>
> Key: FLINK-8955
> URL: https://issues.apache.org/jira/browse/FLINK-8955
> Project: Flink
> Issue Type: Improvement
> Components: Tests
>Affects Versions: 1.5.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Blocker
> Fix For: 1.5.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438492#comment-16438492 ] ASF GitHub Bot commented on FLINK-8955: --- Github user zentol commented on the issue: https://github.com/apache/flink/pull/5780 I would be fine with changing the PR to work against a `MiniCluster` (to drop the JobExecutor changes) and a `TemporaryFolder` that we manage ourselves, i,e. not as a `@Rule` (to drop the icky configuration changes). That's the quickest approach for now. > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[
https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438129#comment-16438129
]
ASF GitHub Bot commented on FLINK-8955:
---
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181534220
--- Diff:
flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResource.java
---
@@ -49,15 +54,17 @@
* Starts a Flink mini cluster as a resource and registers the respective
* ExecutionEnvironment and StreamExecutionEnvironment.
*/
-public class MiniClusterResource extends ExternalResource {
+public class MiniClusterResource extends ExternalResource implements
JobExecutor {
--- End diff --
otherwise we have to revert to using a `MiniCluster` directly, but then we
lose the ´@Rule` benefits.
> Port ClassLoaderITCase to flip6
> ---
>
> Key: FLINK-8955
> URL: https://issues.apache.org/jira/browse/FLINK-8955
> Project: Flink
> Issue Type: Improvement
> Components: Tests
>Affects Versions: 1.5.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Blocker
> Fix For: 1.5.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[
https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438128#comment-16438128
]
ASF GitHub Bot commented on FLINK-8955:
---
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181534097
--- Diff:
flink-tests/src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java
---
@@ -196,21 +188,23 @@ public void
testCheckpointedStreamingClassloaderJobWithCustomClassLoader() throw
// the test also ensures that user specific exceptions are
serializable between JobManager <--> JobClient.
PackagedProgram streamingCheckpointedProg = new
PackagedProgram(new File(STREAMING_CHECKPOINTED_PROG_JAR_FILE));
- TestStreamEnvironment.setAsContext(
- testCluster,
+ TestEnvironment.setAsContext(
+ MINI_CLUSTER_RESOURCE,
parallelism,
Collections.singleton(new
Path(STREAMING_CHECKPOINTED_PROG_JAR_FILE)),
Collections.emptyList());
- // Program should terminate with a 'SuccessException':
- // we can not access the SuccessException here when executing
the tests with maven, because its not available in the jar.
- expectedException.expectCause(
- Matchers.hasProperty("cause",
- hasProperty("class",
- hasProperty("canonicalName", equalTo(
-
"org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException");
-
- streamingCheckpointedProg.invokeInteractiveModeForExecution();
+ try {
+
streamingCheckpointedProg.invokeInteractiveModeForExecution();
+ } catch (Exception e) {
+ // Program should terminate with a 'SuccessException':
+ // we can not access the SuccessException here when
executing the tests with maven, because its not available in the jar.
+ Optional expectedCause =
ExceptionUtils.findThrowable(e,
+ candidate ->
candidate.getClass().getCanonicalName().equals("org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException"));
--- End diff --
Note that this test is a bit funky at the moment; it should not be possible
to access the exception at all since it shouldn't be deserializable. I had one
version where the predicate version of `findThrowble` actually failed since it
couldn't deserialize it, but this issue for some reason disappeared after a
rebase.
> Port ClassLoaderITCase to flip6
> ---
>
> Key: FLINK-8955
> URL: https://issues.apache.org/jira/browse/FLINK-8955
> Project: Flink
> Issue Type: Improvement
> Components: Tests
>Affects Versions: 1.5.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Blocker
> Fix For: 1.5.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[
https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438125#comment-16438125
]
ASF GitHub Bot commented on FLINK-8955:
---
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181533944
--- Diff:
flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResource.java
---
@@ -117,6 +138,7 @@ public TestEnvironment getTestEnvironment() {
@Override
public void before() throws Exception {
+ miniClusterResourceConfiguration =
Preconditions.checkNotNull(miniClusterResourceConfigurationSupplier.get());
--- End diff --
That's not "easy", it doesn't even solve the issue.
The contained `TemporaryFolder` would only be available after the
construction of the `MiniClusterResource`, the folder is however necessary to
create the configuration of the `MiniClusterResource`. Naturally we would like
both of these to be `Rules`, but that's not possible unless the configuration
is generated lazily, as i propose in the PR. Due to the `RuleChain` at the time
that `MiniClusterResource#before` is called we can already access the
`TemporaryFolder`.
The alternative is for the `TemporaryFolder` to be setup in a
`@BeforeClass` method which is then used in the for the configuration. That
would totally works, but it is really unfortunate as the setting of temporary
paths is rather common and required for virtually all tests that use file-based
state backends.
I agree that my proposal is quite hacky, but the question we have to ask is
whether we prefer more complexity in the test vs the `MiniClusterResource`,
where i would prefer the latter.
> Port ClassLoaderITCase to flip6
> ---
>
> Key: FLINK-8955
> URL: https://issues.apache.org/jira/browse/FLINK-8955
> Project: Flink
> Issue Type: Improvement
> Components: Tests
>Affects Versions: 1.5.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Blocker
> Fix For: 1.5.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[
https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438119#comment-16438119
]
ASF GitHub Bot commented on FLINK-8955:
---
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181533175
--- Diff:
flink-tests/src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java
---
@@ -196,21 +188,23 @@ public void
testCheckpointedStreamingClassloaderJobWithCustomClassLoader() throw
// the test also ensures that user specific exceptions are
serializable between JobManager <--> JobClient.
PackagedProgram streamingCheckpointedProg = new
PackagedProgram(new File(STREAMING_CHECKPOINTED_PROG_JAR_FILE));
- TestStreamEnvironment.setAsContext(
- testCluster,
+ TestEnvironment.setAsContext(
+ MINI_CLUSTER_RESOURCE,
parallelism,
Collections.singleton(new
Path(STREAMING_CHECKPOINTED_PROG_JAR_FILE)),
Collections.emptyList());
- // Program should terminate with a 'SuccessException':
- // we can not access the SuccessException here when executing
the tests with maven, because its not available in the jar.
- expectedException.expectCause(
- Matchers.hasProperty("cause",
- hasProperty("class",
- hasProperty("canonicalName", equalTo(
-
"org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException");
-
- streamingCheckpointedProg.invokeInteractiveModeForExecution();
+ try {
+
streamingCheckpointedProg.invokeInteractiveModeForExecution();
+ } catch (Exception e) {
+ // Program should terminate with a 'SuccessException':
+ // we can not access the SuccessException here when
executing the tests with maven, because its not available in the jar.
+ Optional expectedCause =
ExceptionUtils.findThrowable(e,
+ candidate ->
candidate.getClass().getCanonicalName().equals("org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException"));
--- End diff --
Technically that shouldn't be possible. The exception class is part of the
user-jar which is not on the classpath of the test.
> Port ClassLoaderITCase to flip6
> ---
>
> Key: FLINK-8955
> URL: https://issues.apache.org/jira/browse/FLINK-8955
> Project: Flink
> Issue Type: Improvement
> Components: Tests
>Affects Versions: 1.5.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Blocker
> Fix For: 1.5.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[
https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438117#comment-16438117
]
ASF GitHub Bot commented on FLINK-8955:
---
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181532978
--- Diff:
flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResource.java
---
@@ -49,15 +54,17 @@
* Starts a Flink mini cluster as a resource and registers the respective
* ExecutionEnvironment and StreamExecutionEnvironment.
*/
-public class MiniClusterResource extends ExternalResource {
+public class MiniClusterResource extends ExternalResource implements
JobExecutor {
--- End diff --
so that we can pass the MiniClusterResource to
`TestEnvironment.setAsContext()`
> Port ClassLoaderITCase to flip6
> ---
>
> Key: FLINK-8955
> URL: https://issues.apache.org/jira/browse/FLINK-8955
> Project: Flink
> Issue Type: Improvement
> Components: Tests
>Affects Versions: 1.5.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Blocker
> Fix For: 1.5.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[
https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437682#comment-16437682
]
ASF GitHub Bot commented on FLINK-8955:
---
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181467478
--- Diff:
flink-tests/src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java
---
@@ -196,21 +188,23 @@ public void
testCheckpointedStreamingClassloaderJobWithCustomClassLoader() throw
// the test also ensures that user specific exceptions are
serializable between JobManager <--> JobClient.
PackagedProgram streamingCheckpointedProg = new
PackagedProgram(new File(STREAMING_CHECKPOINTED_PROG_JAR_FILE));
- TestStreamEnvironment.setAsContext(
- testCluster,
+ TestEnvironment.setAsContext(
+ MINI_CLUSTER_RESOURCE,
parallelism,
Collections.singleton(new
Path(STREAMING_CHECKPOINTED_PROG_JAR_FILE)),
Collections.emptyList());
- // Program should terminate with a 'SuccessException':
- // we can not access the SuccessException here when executing
the tests with maven, because its not available in the jar.
- expectedException.expectCause(
- Matchers.hasProperty("cause",
- hasProperty("class",
- hasProperty("canonicalName", equalTo(
-
"org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException");
-
- streamingCheckpointedProg.invokeInteractiveModeForExecution();
+ try {
+
streamingCheckpointedProg.invokeInteractiveModeForExecution();
+ } catch (Exception e) {
+ // Program should terminate with a 'SuccessException':
+ // we can not access the SuccessException here when
executing the tests with maven, because its not available in the jar.
+ Optional expectedCause =
ExceptionUtils.findThrowable(e,
+ candidate ->
candidate.getClass().getCanonicalName().equals("org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException"));
--- End diff --
Can't we use `findThrowable(Throwable throwable, Class searchType)`?
> Port ClassLoaderITCase to flip6
> ---
>
> Key: FLINK-8955
> URL: https://issues.apache.org/jira/browse/FLINK-8955
> Project: Flink
> Issue Type: Improvement
> Components: Tests
>Affects Versions: 1.5.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Blocker
> Fix For: 1.5.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[
https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437679#comment-16437679
]
ASF GitHub Bot commented on FLINK-8955:
---
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181466728
--- Diff:
flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResource.java
---
@@ -117,6 +138,7 @@ public TestEnvironment getTestEnvironment() {
@Override
public void before() throws Exception {
+ miniClusterResourceConfiguration =
Preconditions.checkNotNull(miniClusterResourceConfigurationSupplier.get());
--- End diff --
I think we are mixing concerns by letting the `MiniClusterResource` change
its behaviour across multiple tests. I think we should not pass in a
configuration supplier which can lazily instantiate a configuration. Instead if
you need a different `MiniClusterResource` per test, then the test should
instantiate the respective `MiniClusterResource` with the proper configuration.
> Port ClassLoaderITCase to flip6
> ---
>
> Key: FLINK-8955
> URL: https://issues.apache.org/jira/browse/FLINK-8955
> Project: Flink
> Issue Type: Improvement
> Components: Tests
>Affects Versions: 1.5.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Blocker
> Fix For: 1.5.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[
https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437680#comment-16437680
]
ASF GitHub Bot commented on FLINK-8955:
---
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181465966
--- Diff:
flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResource.java
---
@@ -225,6 +247,11 @@ private void startMiniCluster() throws Exception {
}
}
+ @Override
+ public JobExecutionResult executeJobBlocking(JobGraph jobGraph) throws
JobExecutionException, InterruptedException {
--- End diff --
I think we should use the `MiniClusterClient` for job submission. The
`JobExecutor` interface is only an internal interface to bridge between the two
different mini clusters.
> Port ClassLoaderITCase to flip6
> ---
>
> Key: FLINK-8955
> URL: https://issues.apache.org/jira/browse/FLINK-8955
> Project: Flink
> Issue Type: Improvement
> Components: Tests
>Affects Versions: 1.5.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Blocker
> Fix For: 1.5.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[
https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437681#comment-16437681
]
ASF GitHub Bot commented on FLINK-8955:
---
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181467950
--- Diff:
flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResource.java
---
@@ -117,6 +138,7 @@ public TestEnvironment getTestEnvironment() {
@Override
public void before() throws Exception {
+ miniClusterResourceConfiguration =
Preconditions.checkNotNull(miniClusterResourceConfigurationSupplier.get());
--- End diff --
The same applies to rule chains. An easy solution would be to create a
`MiniClusterWithTemporaryFolderResource` which encapsulates this logic.
> Port ClassLoaderITCase to flip6
> ---
>
> Key: FLINK-8955
> URL: https://issues.apache.org/jira/browse/FLINK-8955
> Project: Flink
> Issue Type: Improvement
> Components: Tests
>Affects Versions: 1.5.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Blocker
> Fix For: 1.5.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[
https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437683#comment-16437683
]
ASF GitHub Bot commented on FLINK-8955:
---
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181468061
--- Diff:
flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResource.java
---
@@ -49,15 +54,17 @@
* Starts a Flink mini cluster as a resource and registers the respective
* ExecutionEnvironment and StreamExecutionEnvironment.
*/
-public class MiniClusterResource extends ExternalResource {
+public class MiniClusterResource extends ExternalResource implements
JobExecutor {
--- End diff --
Why do we need to implement `JobExecutor`?
> Port ClassLoaderITCase to flip6
> ---
>
> Key: FLINK-8955
> URL: https://issues.apache.org/jira/browse/FLINK-8955
> Project: Flink
> Issue Type: Improvement
> Components: Tests
>Affects Versions: 1.5.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Blocker
> Fix For: 1.5.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-8955) Port ClassLoaderITCase to flip6
[ https://issues.apache.org/jira/browse/FLINK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437667#comment-16437667 ] ASF GitHub Bot commented on FLINK-8955: --- Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5780 The `ClassLoaderITCase` fails on Travis. There are also checkstyle violations: ``` 10:04:01.988 [ERROR] src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java:[62,15] (imports) UnusedImports: Unused import: org.junit.Assert.assertEquals. 10:04:01.988 [ERROR] src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java:[65,15] (imports) UnusedImports: Unused import: org.junit.Assert.fail. ``` > Port ClassLoaderITCase to flip6 > --- > > Key: FLINK-8955 > URL: https://issues.apache.org/jira/browse/FLINK-8955 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.5.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
