[jira] [Commented] (FLINK-6836) Failing YARNSessionCapacitySchedulerITCase.testTaskManagerFailure
[
https://issues.apache.org/jira/browse/FLINK-6836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16056949#comment-16056949
]
ASF GitHub Bot commented on FLINK-6836:
---
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/4120
> Failing YARNSessionCapacitySchedulerITCase.testTaskManagerFailure
> -
>
> Key: FLINK-6836
> URL: https://issues.apache.org/jira/browse/FLINK-6836
> Project: Flink
> Issue Type: Bug
> Components: Tests, YARN
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Critical
> Labels: test-stability
>
> The master is currently unstable. The
> {{YARNSessionCapacitySchedulerITCase.testTaskManagerFailure}} fails with
> Hadoop version {{2.6.5}}, {{2.7.3}} and {{2.8.0}}.
> See this build [1] for example.
> [1] https://travis-ci.org/apache/flink/builds/238720589
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-6836) Failing YARNSessionCapacitySchedulerITCase.testTaskManagerFailure
[
https://issues.apache.org/jira/browse/FLINK-6836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053579#comment-16053579
]
ASF GitHub Bot commented on FLINK-6836:
---
Github user tillrohrmann commented on the issue:
https://github.com/apache/flink/pull/4120
Thanks for the review @zentol. Failing test case is unrelated and fixed via
#4139. Merging this PR.
> Failing YARNSessionCapacitySchedulerITCase.testTaskManagerFailure
> -
>
> Key: FLINK-6836
> URL: https://issues.apache.org/jira/browse/FLINK-6836
> Project: Flink
> Issue Type: Bug
> Components: Tests, YARN
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Critical
> Labels: test-stability
>
> The master is currently unstable. The
> {{YARNSessionCapacitySchedulerITCase.testTaskManagerFailure}} fails with
> Hadoop version {{2.6.5}}, {{2.7.3}} and {{2.8.0}}.
> See this build [1] for example.
> [1] https://travis-ci.org/apache/flink/builds/238720589
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-6836) Failing YARNSessionCapacitySchedulerITCase.testTaskManagerFailure
[
https://issues.apache.org/jira/browse/FLINK-6836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050495#comment-16050495
]
ASF GitHub Bot commented on FLINK-6836:
---
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/4120
+1 to merge.
> Failing YARNSessionCapacitySchedulerITCase.testTaskManagerFailure
> -
>
> Key: FLINK-6836
> URL: https://issues.apache.org/jira/browse/FLINK-6836
> Project: Flink
> Issue Type: Bug
> Components: Tests, YARN
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Critical
> Labels: test-stability
>
> The master is currently unstable. The
> {{YARNSessionCapacitySchedulerITCase.testTaskManagerFailure}} fails with
> Hadoop version {{2.6.5}}, {{2.7.3}} and {{2.8.0}}.
> See this build [1] for example.
> [1] https://travis-ci.org/apache/flink/builds/238720589
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-6836) Failing YARNSessionCapacitySchedulerITCase.testTaskManagerFailure
[
https://issues.apache.org/jira/browse/FLINK-6836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050238#comment-16050238
]
ASF GitHub Bot commented on FLINK-6836:
---
Github user tillrohrmann commented on the issue:
https://github.com/apache/flink/pull/4120
I first thought I had to downgrade, because of the aforementioned problem.
However, Hadoop 2.8.0 needs at least httpclient `4.5.2`. Luckily, they fixed
the problem with the `URLEncodedUtils#parse` method in `4.5.3`. Therefore I
bumped the version again.
Concerning moving the dependency management to `flink-shaded-hadoop`: I
think this is unfortunately not possible since due the immutable reactor builds
from maven, `flink-yarn-tests` pulls all dependencies from
`flink-shaded-hadoop` and `flink-shaded-yarn-tests`. This also means that
depending on the classpath order, we either use the shaded Yarn mini-cluster or
not. In the latter case we have to make sure that we are using `httpclient`
`4.5.3` which is only the case if we define the dependency management in the
parent pom.
As a side remark, given this situation with maven, I'm not sure whether we
actually need the `flink-shaded-yarn-tests` module.
A proper solution would be fixing the problem with the immutable builds.
For example, everything works properly if we first install the shaded jars and
then build the flink-yarn-tests. Then maven will properly resolve all
dependencies. In this case, I think it should be possible to move the
dependency management into `flink-shaded-hadoop`.
> Failing YARNSessionCapacitySchedulerITCase.testTaskManagerFailure
> -
>
> Key: FLINK-6836
> URL: https://issues.apache.org/jira/browse/FLINK-6836
> Project: Flink
> Issue Type: Bug
> Components: Tests, YARN
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Critical
> Labels: test-stability
>
> The master is currently unstable. The
> {{YARNSessionCapacitySchedulerITCase.testTaskManagerFailure}} fails with
> Hadoop version {{2.6.5}}, {{2.7.3}} and {{2.8.0}}.
> See this build [1] for example.
> [1] https://travis-ci.org/apache/flink/builds/238720589
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-6836) Failing YARNSessionCapacitySchedulerITCase.testTaskManagerFailure
[
https://issues.apache.org/jira/browse/FLINK-6836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050134#comment-16050134
]
ASF GitHub Bot commented on FLINK-6836:
---
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/4120
The PR descriptions says that you're downgrading the dependency versions,
but with the second commit your actually upgrading them.
Also, couldn't we move the dependency management into `flink-shaded-hadoop`?
> Failing YARNSessionCapacitySchedulerITCase.testTaskManagerFailure
> -
>
> Key: FLINK-6836
> URL: https://issues.apache.org/jira/browse/FLINK-6836
> Project: Flink
> Issue Type: Bug
> Components: Tests, YARN
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Critical
> Labels: test-stability
>
> The master is currently unstable. The
> {{YARNSessionCapacitySchedulerITCase.testTaskManagerFailure}} fails with
> Hadoop version {{2.6.5}}, {{2.7.3}} and {{2.8.0}}.
> See this build [1] for example.
> [1] https://travis-ci.org/apache/flink/builds/238720589
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-6836) Failing YARNSessionCapacitySchedulerITCase.testTaskManagerFailure
[
https://issues.apache.org/jira/browse/FLINK-6836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049339#comment-16049339
]
ASF GitHub Bot commented on FLINK-6836:
---
Github user tillrohrmann commented on the issue:
https://github.com/apache/flink/pull/4120
Rebasing onto the latest master
> Failing YARNSessionCapacitySchedulerITCase.testTaskManagerFailure
> -
>
> Key: FLINK-6836
> URL: https://issues.apache.org/jira/browse/FLINK-6836
> Project: Flink
> Issue Type: Bug
> Components: Tests, YARN
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Critical
> Labels: test-stability
>
> The master is currently unstable. The
> {{YARNSessionCapacitySchedulerITCase.testTaskManagerFailure}} fails with
> Hadoop version {{2.6.5}}, {{2.7.3}} and {{2.8.0}}.
> See this build [1] for example.
> [1] https://travis-ci.org/apache/flink/builds/238720589
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-6836) Failing YARNSessionCapacitySchedulerITCase.testTaskManagerFailure
[
https://issues.apache.org/jira/browse/FLINK-6836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048440#comment-16048440
]
ASF GitHub Bot commented on FLINK-6836:
---
GitHub user tillrohrmann opened a pull request:
https://github.com/apache/flink/pull/4120
[FLINK-6836] [tests] Fix YARNSessionCapacitySchedulerITCase to work with
Hadoop 2.6.5, 2.7.3 and 2.8.0
Due to MNG-5899, maven cannot resolve dependency reduced poms in a multi
project build. Therefore, flink-yarn-tests pulls in a wrong version of
org.apache.httpcomponents.httpclient which does not work with Hadoop's
ServletUtils together. As a solution we have to move the dependency management
for the httpclient and httpcore version into the parent pom.xml.
Another problem is the version of these libraries which has been recently
bumped. In 4.4, httpclient changed its behaviour such that
URLEncodedUtils#parse(String, Charset) now throws a NPE if the first parameter
is null. In 4.2.6, an empty list was returned instead. Due to this
incompatibility, we reverted the change and set the version to its previous
value.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tillrohrmann/flink fixYarnSessionTest
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4120.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4120
commit 37d5577cd12454ee3b5aa39206989bb89faf0cf8
Author: Till Rohrmann
Date: 2017-06-13T21:26:10Z
[FLINK-6836] [tests] Fix YARNSessionCapacitySchedulerITCase to work with
Hadoop 2.6.5, 2.7.3 and 2.8.0
Due to MNG-5899, maven cannot resolve dependency reduced poms in a multi
project build. Therefore,
flink-yarn-tests pulls in a wrong version of
org.apache.httpcomponents.httpclient which does not work
with Hadoop's ServletUtils together. As a solution we have to move the
dependency management for the
httpclient and httpcore version into the parent pom.xml.
Another problem is the version of these libraries which has been recently
bumped. In 4.4, httpclient
changed its behaviour such that URLEncodedUtils#parse(String, Charset) now
throws a NPE if the first
parameter is null. In 4.2.6, an empty list was returned instead. Due to
this incompatibility, we reverted
the change and set the version to its previous value.
> Failing YARNSessionCapacitySchedulerITCase.testTaskManagerFailure
> -
>
> Key: FLINK-6836
> URL: https://issues.apache.org/jira/browse/FLINK-6836
> Project: Flink
> Issue Type: Bug
> Components: Tests, YARN
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Critical
> Labels: test-stability
>
> The master is currently unstable. The
> {{YARNSessionCapacitySchedulerITCase.testTaskManagerFailure}} fails with
> Hadoop version {{2.6.5}}, {{2.7.3}} and {{2.8.0}}.
> See this build [1] for example.
> [1] https://travis-ci.org/apache/flink/builds/238720589
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-6836) Failing YARNSessionCapacitySchedulerITCase.testTaskManagerFailure
[
https://issues.apache.org/jira/browse/FLINK-6836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16036176#comment-16036176
]
Jark Wu commented on FLINK-6836:
And also
{code}
Failed tests:
YARNSessionCapacitySchedulerITCase>YarnTestBase.checkClusterEmpty:199 There is
at least one application on the cluster is not finished.App
application_1496510462161_0006 is in state RUNNING
{code}
> Failing YARNSessionCapacitySchedulerITCase.testTaskManagerFailure
> -
>
> Key: FLINK-6836
> URL: https://issues.apache.org/jira/browse/FLINK-6836
> Project: Flink
> Issue Type: Bug
> Components: Tests, YARN
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Critical
> Labels: test-stability
>
> The master is currently unstable. The
> {{YARNSessionCapacitySchedulerITCase.testTaskManagerFailure}} fails with
> Hadoop version {{2.6.5}}, {{2.7.3}} and {{2.8.0}}.
> See this build [1] for example.
> [1] https://travis-ci.org/apache/flink/builds/238720589
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
