[jira] [Commented] (YARN-1042) add ability to specify affinity/anti-affinity in container requests
[ https://issues.apache.org/jira/browse/YARN-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539467#comment-14539467 ] Yang Weiwei commented on YARN-1042: --- There is no update for a long time, anything new here ? This looks like a nice feature that should be helping us a lot, is this a correct direction that we should put some more efforts to get this done in RM side ? I noticed there are alternatives in slider project, e.g SLIDER-82. Please advise. > add ability to specify affinity/anti-affinity in container requests > --- > > Key: YARN-1042 > URL: https://issues.apache.org/jira/browse/YARN-1042 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Affects Versions: 3.0.0 >Reporter: Steve Loughran >Assignee: Arun C Murthy > Attachments: YARN-1042-demo.patch > > > container requests to the AM should be able to request anti-affinity to > ensure that things like Region Servers don't come up on the same failure > zones. > Similarly, you may be able to want to specify affinity to same host or rack > without specifying which specific host/rack. Example: bringing up a small > giraph cluster in a large YARN cluster would benefit from having the > processes in the same rack purely for bandwidth reasons. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2605) [RM HA] Rest api endpoints doing redirect incorrectly
[ https://issues.apache.org/jira/browse/YARN-2605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539341#comment-14539341 ] Yang Weiwei commented on YARN-2605: --- Hello [~xgong] I noticed that you set an Ignore tag for testRMWebAppRedirect in your patch, can you please let me know why to ignore this test case ? Any idea that how to fix this ? I opened another JIRA YARN-3601, please let me know if that one is valid. > [RM HA] Rest api endpoints doing redirect incorrectly > - > > Key: YARN-2605 > URL: https://issues.apache.org/jira/browse/YARN-2605 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Affects Versions: 2.4.0 >Reporter: bc Wong >Assignee: Xuan Gong > Labels: newbie > Fix For: 2.7.1 > > Attachments: YARN-2605.1.patch, YARN-2605.2.patch > > > The standby RM's webui tries to do a redirect via meta-refresh. That is fine > for pages designed to be viewed by web browsers. But the API endpoints > shouldn't do that. Most programmatic HTTP clients do not do meta-refresh. I'd > suggest HTTP 303, or return a well-defined error message (json or xml) > stating that the standby status and a link to the active RM. > The standby RM is returning this today: > {noformat} > $ curl -i http://bcsec-1.ent.cloudera.com:8088/ws/v1/cluster/metrics > HTTP/1.1 200 OK > Cache-Control: no-cache > Expires: Thu, 25 Sep 2014 18:34:53 GMT > Date: Thu, 25 Sep 2014 18:34:53 GMT > Pragma: no-cache > Expires: Thu, 25 Sep 2014 18:34:53 GMT > Date: Thu, 25 Sep 2014 18:34:53 GMT > Pragma: no-cache > Content-Type: text/plain; charset=UTF-8 > Refresh: 3; url=http://bcsec-2.ent.cloudera.com:8088/ws/v1/cluster/metrics > Content-Length: 117 > Server: Jetty(6.1.26) > This is standby RM. Redirecting to the current active RM: > http://bcsec-2.ent.cloudera.com:8088/ws/v1/cluster/metrics > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3526) ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster
[ https://issues.apache.org/jira/browse/YARN-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Weiwei updated YARN-3526: -- Attachment: YARN-3526.002.patch Fixed trailing white spaces > ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster > - > > Key: YARN-3526 > URL: https://issues.apache.org/jira/browse/YARN-3526 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, webapp >Affects Versions: 2.6.0 > Environment: Red Hat Enterprise Linux Server 6.4 >Reporter: Yang Weiwei >Assignee: Yang Weiwei > Labels: BB2015-05-TBR > Attachments: YARN-3526.001.patch, YARN-3526.002.patch > > > On a QJM HA cluster, view RM web UI to track job status, it shows > This is standby RM. Redirecting to the current active RM: > http://:8088/proxy/application_1427338037905_0008/mapreduce > it refreshes every 3 sec but never going to the correct tracking page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3526) ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster
[ https://issues.apache.org/jira/browse/YARN-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14534640#comment-14534640 ] Yang Weiwei commented on YARN-3526: --- Hello [~steve_l] and [~vinodkv] I uploaded a new patch, I added test case into existing class TestRMFailover.testRMWebAppRedirect, where I believe that is the best place for it. However I found this method is marked as "Ignore" since YARN-2605 was committed. I tried to run the test case by reverting the changes YARN-2605 made, it works fine. But with the changes, I got NPEs. I created another issue to fix this unit test, need to discuss with [~xgong]. The JIRA number is YARN-3601. > ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster > - > > Key: YARN-3526 > URL: https://issues.apache.org/jira/browse/YARN-3526 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, webapp >Affects Versions: 2.6.0 > Environment: Red Hat Enterprise Linux Server 6.4 >Reporter: Yang Weiwei >Assignee: Yang Weiwei > Labels: BB2015-05-TBR > Attachments: YARN-3526.001.patch > > > On a QJM HA cluster, view RM web UI to track job status, it shows > This is standby RM. Redirecting to the current active RM: > http://:8088/proxy/application_1427338037905_0008/mapreduce > it refreshes every 3 sec but never going to the correct tracking page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3601) Fix UT TestRMFailover.testRMWebAppRedirect
[ https://issues.apache.org/jira/browse/YARN-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Weiwei updated YARN-3601: -- Assignee: (was: Yang Weiwei) > Fix UT TestRMFailover.testRMWebAppRedirect > -- > > Key: YARN-3601 > URL: https://issues.apache.org/jira/browse/YARN-3601 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, webapp > Environment: Red Hat Enterprise Linux Workstation release 6.5 > (Santiago) >Reporter: Yang Weiwei >Priority: Critical > Labels: BB2015-05-TBR, test > > This test case was not working since the commit from YARN-2605. It failed > with NPE exception. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3601) Fix UT TestRMFailover.testRMWebAppRedirect
[ https://issues.apache.org/jira/browse/YARN-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14534638#comment-14534638 ] Yang Weiwei commented on YARN-3601: --- YARN-2605 changes the response header info in RMWebAppFilter, it used to use Refresh but now it is Location. Need to revise the test case to provide test coverage for webapp redirection. > Fix UT TestRMFailover.testRMWebAppRedirect > -- > > Key: YARN-3601 > URL: https://issues.apache.org/jira/browse/YARN-3601 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, webapp > Environment: Red Hat Enterprise Linux Workstation release 6.5 > (Santiago) >Reporter: Yang Weiwei >Assignee: Yang Weiwei >Priority: Critical > Labels: BB2015-05-TBR, test > > This test case was not working since the commit from YARN-2605. It failed > with NPE exception. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3601) Fix UT TestRMFailover.testRMWebAppRedirect
[ https://issues.apache.org/jira/browse/YARN-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Weiwei updated YARN-3601: -- Description: This test case was not working since the commit from YARN-2605. It failed with NPE exception. (was: This test case was not working since the commit from YARN-2605. ) > Fix UT TestRMFailover.testRMWebAppRedirect > -- > > Key: YARN-3601 > URL: https://issues.apache.org/jira/browse/YARN-3601 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, webapp > Environment: Red Hat Enterprise Linux Workstation release 6.5 > (Santiago) >Reporter: Yang Weiwei >Assignee: Yang Weiwei >Priority: Critical > Labels: BB2015-05-TBR, test > > This test case was not working since the commit from YARN-2605. It failed > with NPE exception. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (YARN-3601) Fix UT TestRMFailover.testRMWebAppRedirect
Yang Weiwei created YARN-3601: - Summary: Fix UT TestRMFailover.testRMWebAppRedirect Key: YARN-3601 URL: https://issues.apache.org/jira/browse/YARN-3601 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager, webapp Environment: Red Hat Enterprise Linux Workstation release 6.5 (Santiago) Reporter: Yang Weiwei Assignee: Yang Weiwei Priority: Critical This test case was not working since the commit from YARN-2605. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3526) ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster
[ https://issues.apache.org/jira/browse/YARN-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Weiwei updated YARN-3526: -- Attachment: YARN-3526.001.patch > ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster > - > > Key: YARN-3526 > URL: https://issues.apache.org/jira/browse/YARN-3526 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, webapp >Affects Versions: 2.6.0 > Environment: Red Hat Enterprise Linux Server 6.4 >Reporter: Yang Weiwei >Assignee: Yang Weiwei > Labels: BB2015-05-TBR > Attachments: YARN-3526.001.patch > > > On a QJM HA cluster, view RM web UI to track job status, it shows > This is standby RM. Redirecting to the current active RM: > http://:8088/proxy/application_1427338037905_0008/mapreduce > it refreshes every 3 sec but never going to the correct tracking page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3526) ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster
[ https://issues.apache.org/jira/browse/YARN-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Weiwei updated YARN-3526: -- Attachment: (was: YARN-3526.patch) > ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster > - > > Key: YARN-3526 > URL: https://issues.apache.org/jira/browse/YARN-3526 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, webapp >Affects Versions: 2.6.0 > Environment: Red Hat Enterprise Linux Server 6.4 >Reporter: Yang Weiwei >Assignee: Yang Weiwei > Labels: BB2015-05-TBR > > On a QJM HA cluster, view RM web UI to track job status, it shows > This is standby RM. Redirecting to the current active RM: > http://:8088/proxy/application_1427338037905_0008/mapreduce > it refreshes every 3 sec but never going to the correct tracking page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3526) ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster
[ https://issues.apache.org/jira/browse/YARN-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14530133#comment-14530133 ] Yang Weiwei commented on YARN-3526: --- Hi Vinod Sure let me add a test case. > ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster > - > > Key: YARN-3526 > URL: https://issues.apache.org/jira/browse/YARN-3526 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, webapp >Affects Versions: 2.6.0 > Environment: Red Hat Enterprise Linux Server 6.4 >Reporter: Yang Weiwei >Assignee: Yang Weiwei > Labels: BB2015-05-TBR > Attachments: YARN-3526.patch > > > On a QJM HA cluster, view RM web UI to track job status, it shows > This is standby RM. Redirecting to the current active RM: > http://:8088/proxy/application_1427338037905_0008/mapreduce > it refreshes every 3 sec but never going to the correct tracking page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3526) ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster
[ https://issues.apache.org/jira/browse/YARN-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Weiwei updated YARN-3526: -- Attachment: YARN-3526.patch > ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster > - > > Key: YARN-3526 > URL: https://issues.apache.org/jira/browse/YARN-3526 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, webapp >Affects Versions: 2.6.0 > Environment: Red Hat Enterprise Linux Server 6.4 >Reporter: Yang Weiwei > Attachments: YARN-3526.patch > > > On a QJM HA cluster, view RM web UI to track job status, it shows > This is standby RM. Redirecting to the current active RM: > http://:8088/proxy/application_1427338037905_0008/mapreduce > it refreshes every 3 sec but never going to the correct tracking page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (YARN-3526) ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster
[ https://issues.apache.org/jira/browse/YARN-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Weiwei reopened YARN-3526: --- I don't think this is a duplicate of YARN-2605. YARN-2605 fixes the issues in REST API to avoid meta-refresh. But this JIRA is about the issue when application master proxy server runs on the standby RM, the traffic to the proxy server gets incorrectly redirected. I have attached a patch to fix this. Please help to review. > ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster > - > > Key: YARN-3526 > URL: https://issues.apache.org/jira/browse/YARN-3526 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, webapp >Affects Versions: 2.6.0 > Environment: Red Hat Enterprise Linux Server 6.4 >Reporter: Yang Weiwei > > On a QJM HA cluster, view RM web UI to track job status, it shows > This is standby RM. Redirecting to the current active RM: > http://:8088/proxy/application_1427338037905_0008/mapreduce > it refreshes every 3 sec but never going to the correct tracking page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (YARN-3526) ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster
Yang Weiwei created YARN-3526: - Summary: ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster Key: YARN-3526 URL: https://issues.apache.org/jira/browse/YARN-3526 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager, webapp Affects Versions: 2.6.0 Environment: Red Hat Enterprise Linux Server 6.4 Reporter: Yang Weiwei On a QJM HA cluster, view RM web UI to track job status, it shows This is standby RM. Redirecting to the current active RM: http://:8088/proxy/application_1427338037905_0008/mapreduce it refreshes every 3 sec but never going to the correct tracking page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3193) When visit standby RM webui, it will redirect to the active RM webui slowly.
[ https://issues.apache.org/jira/browse/YARN-3193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14381285#comment-14381285 ] Yang Weiwei commented on YARN-3193: --- The redirect seems to happen in 3 seconds, if that what you means by slow. > When visit standby RM webui, it will redirect to the active RM webui slowly. > > > Key: YARN-3193 > URL: https://issues.apache.org/jira/browse/YARN-3193 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp >Reporter: Japs_123 >Priority: Minor > > when visit the standby RM web ui, it will redirect to the active RM web ui. > but this redirect is very slow which give client bad experience. I have try > to visit standby namenode, it directly show the web to client quickly. So, > can we improve this experience with YARN like HDFS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)