[jira] [Updated] (MESOS-7908) Mesos UI Keeps refreshing
[ https://issues.apache.org/jira/browse/MESOS-7908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lax updated MESOS-7908: --- Attachment: mesos_ui_refresh.png Attachment shows the UI refresh issue. > Mesos UI Keeps refreshing > - > > Key: MESOS-7908 > URL: https://issues.apache.org/jira/browse/MESOS-7908 > Project: Mesos > Issue Type: Bug >Affects Versions: 1.2.0, 1.3.0, 1.3.1 >Reporter: Lax >Priority: Critical > Attachments: mesos_ui_refresh.png > > > I tried to upgrade mesos from 1.0.1 to latest and suddenly I see Mesos UI > keeps refreshing every few seconds. I can say the issue remains at least from > version 1.1.x onwards. > The UI refresh will be seen only when you access Mesos thru proxy ports > (other than 5050) or custom DNS. > Root cause for this appears to be to be how mesos ui is constructing the end > point in here: > https://github.com/apache/mesos/blob/master/src/webui/master/static/js/controllers.js#L362-L365. > It basically learning mesos host and port from mesos state API response and > using that to load in the UI. But what you learn will be the internal host > name(pretty much node name) and port (5050) and it wont work in cases where > user proxy's the mesos port eg: 9443 in my case and also have custom DNS > format eg: *mesos-laxmesos4.dev.io* > So basically when I load mesos ui with URI: > https://mesos-laxmesos4.dev.io:9443, above pointed API is returning > https://[my node name]:5050/metrics/... while expected path is > https://mesos-laxmesos4.dev.io:9443/metrics/... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MESOS-7908) Mesos UI Keeps refreshing
[ https://issues.apache.org/jira/browse/MESOS-7908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16137672#comment-16137672 ] Lax commented on MESOS-7908: Note: Issue goes away if I comment out the https://github.com/apache/mesos/blob/master/src/webui/master/static/js/controllers.js#L362-L365. Will submit in a PR with this fix. > Mesos UI Keeps refreshing > - > > Key: MESOS-7908 > URL: https://issues.apache.org/jira/browse/MESOS-7908 > Project: Mesos > Issue Type: Bug >Affects Versions: 1.2.0, 1.3.0, 1.3.1 >Reporter: Lax >Priority: Critical > > I tried to upgrade mesos from 1.0.1 to latest and suddenly I see Mesos UI > keeps refreshing every few seconds. I can say the issue remains at least from > version 1.1.x onwards. > The UI refresh will be seen only when you access Mesos thru proxy ports > (other than 5050) or custom DNS. > Root cause for this appears to be to be how mesos ui is constructing the end > point in here: > https://github.com/apache/mesos/blob/master/src/webui/master/static/js/controllers.js#L362-L365. > It basically learning mesos host and port from mesos state API response and > using that to load in the UI. But what you learn will be the internal host > name(pretty much node name) and port (5050) and it wont work in cases where > user proxy's the mesos port eg: 9443 in my case and also have custom DNS > format eg: *mesos-laxmesos4.dev.io* > So basically when I load mesos ui with URI: > https://mesos-laxmesos4.dev.io:9443, above pointed API is returning > https://[my node name]:5050/metrics/... while expected path is > https://mesos-laxmesos4.dev.io:9443/metrics/... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (MESOS-7908) Mesos UI Keeps refreshing
Lax created MESOS-7908: -- Summary: Mesos UI Keeps refreshing Key: MESOS-7908 URL: https://issues.apache.org/jira/browse/MESOS-7908 Project: Mesos Issue Type: Bug Affects Versions: 1.3.1, 1.3.0, 1.2.0 Reporter: Lax Priority: Critical I tried to upgrade mesos from 1.0.1 to latest and suddenly I see Mesos UI keeps refreshing every few seconds. I can say the issue remains at least from version 1.1.x onwards. The UI refresh will be seen only when you access Mesos thru proxy ports (other than 5050) or custom DNS. Root cause for this appears to be to be how mesos ui is constructing the end point in here: https://github.com/apache/mesos/blob/master/src/webui/master/static/js/controllers.js#L362-L365. It basically learning mesos host and port from mesos state API response and using that to load in the UI. But what you learn will be the internal host name(pretty much node name) and port (5050) and it wont work in cases where user proxy's the mesos port eg: 9443 in my case and also have custom DNS format eg: *mesos-laxmesos4.dev.io* So basically when I load mesos ui with URI: https://mesos-laxmesos4.dev.io:9443, above pointed API is returning https://[my node name]:5050/metrics/... while expected path is https://mesos-laxmesos4.dev.io:9443/metrics/... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (MESOS-6649) Mesos build fails when run configure with network-isolator
Lax created MESOS-6649: -- Summary: Mesos build fails when run configure with network-isolator Key: MESOS-6649 URL: https://issues.apache.org/jira/browse/MESOS-6649 Project: Mesos Issue Type: Bug Reporter: Lax '../configure --with-network-isolator' is failing with error(s) on Centos 7.2 distributions. Issues seen in order 1. Please install libnl3 (version 3.2.26 or higher ) Issue is there is none rpms available with version > 3.2.21 for Centos 7 distribution. Only workaround was to download libnl3 from https://github.com/thom311/libnl and install it manually 2. After the above fix, ran into 'We need CRAM-MD5 support for SASL authentication' on rerunning the configure with network isolatoer. However the issue was not seen if just configure was run -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5770) Mesos state api reporting Host IP instead of container IP with health check
[ https://issues.apache.org/jira/browse/MESOS-5770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383248#comment-15383248 ] Lax commented on MESOS-5770: You can mark it resolved now. Have tried out the fix (https://github.com/apache/mesos/commit/cca92ceac75c7639c4ae93660b3bb15cce11c5c3) and it works. Now I can learn container IP with health check on too - Thanks > Mesos state api reporting Host IP instead of container IP with health check > > > Key: MESOS-5770 > URL: https://issues.apache.org/jira/browse/MESOS-5770 > Project: Mesos > Issue Type: Bug >Reporter: Lax >Priority: Critical > Fix For: 0.28.0 > > > Am using Mesos IP per container using docker containerizer (via Calico). > Mesos state API (/master/state.json) seems to report container IP as long as > I have no health check on my task. As soon as I add health check to the task, > mesos start reporting Host IP instead of Container IP. > Had initially opened this bug on Marathon > (https://github.com/mesosphere/marathon/issues/3907). But then told the issue > is with Mesos reporting the wrong IP. > Here are versions of Mesos and Marathon I was using. > Mesos: 0.28.0.2 > Marathon: 0.15.3-1.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5294) Status updates after a health check are incomplete or invalid
[
https://issues.apache.org/jira/browse/MESOS-5294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15379716#comment-15379716
]
Lax commented on MESOS-5294:
Issue seems happening much before that, can't seem to figure out where (due to
lack of my C++ expertise). ContainerStatus object wherever it was learned in
the API trail(hoping thru docker inspect) should have learned about the
Container's Internal IP info. Since it was missing, the above API ends up
populating it with Host IP.
> Status updates after a health check are incomplete or invalid
> -
>
> Key: MESOS-5294
> URL: https://issues.apache.org/jira/browse/MESOS-5294
> Project: Mesos
> Issue Type: Bug
> Environment: mesos 0.28.0, docker 1.11, marathon 0.15.3, mesos-dns,
> ubuntu 14.04
>Reporter: Travis Hegner
>Assignee: haosdent
>
> With command health checks enabled via marathon, mesos-dns will resolve the
> task correctly until the task is reported as "healthy". At that point,
> mesos-dns stops resolving the task correctly.
> -Digging through src/docker/executor.cpp, I found that in the
> {{taskHealthUpdated()}} function is attempting to copy the taskID to the new
> status instance with-
> {code}status.mutable_task_id()->CopyFrom(taskID);{code}
> -but other instances of status updates have a similar line-
> {code}status.mutable_task_id()->CopyFrom(taskID.get());{code}
> -My assumption is that this difference is causing the status update after a
> health check to not have a proper taskID, which in turn is causing an
> incorrect state.json output.-
> -I'll try to get a patch together soon.-
> UPDATE:
> None of the above assumption are correct. Something else is causing the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-5770) Mesos state api reporting Host IP instead of container IP with health check
[ https://issues.apache.org/jira/browse/MESOS-5770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366368#comment-15366368 ] Lax commented on MESOS-5770: Between I tried packaging the latest code base still see the issue. Any pointers on where I can start digging up while you are also looking into it. We badly need this resolved for us to move forward. > Mesos state api reporting Host IP instead of container IP with health check > > > Key: MESOS-5770 > URL: https://issues.apache.org/jira/browse/MESOS-5770 > Project: Mesos > Issue Type: Bug >Reporter: Lax >Priority: Critical > Fix For: 0.28.0 > > > Am using Mesos IP per container using docker containerizer (via Calico). > Mesos state API (/master/state.json) seems to report container IP as long as > I have no health check on my task. As soon as I add health check to the task, > mesos start reporting Host IP instead of Container IP. > Had initially opened this bug on Marathon > (https://github.com/mesosphere/marathon/issues/3907). But then told the issue > is with Mesos reporting the wrong IP. > Here are versions of Mesos and Marathon I was using. > Mesos: 0.28.0.2 > Marathon: 0.15.3-1.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5770) Mesos state api reporting Host IP instead of container IP with health check
[ https://issues.apache.org/jira/browse/MESOS-5770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362862#comment-15362862 ] Lax commented on MESOS-5770: Ok, thanks haosdent. > Mesos state api reporting Host IP instead of container IP with health check > > > Key: MESOS-5770 > URL: https://issues.apache.org/jira/browse/MESOS-5770 > Project: Mesos > Issue Type: Bug >Reporter: Lax >Priority: Critical > Fix For: 0.28.0 > > > Am using Mesos IP per container using docker containerizer (via Calico). > Mesos state API (/master/state.json) seems to report container IP as long as > I have no health check on my task. As soon as I add health check to the task, > mesos start reporting Host IP instead of Container IP. > Had initially opened this bug on Marathon > (https://github.com/mesosphere/marathon/issues/3907). But then told the issue > is with Mesos reporting the wrong IP. > Here are versions of Mesos and Marathon I was using. > Mesos: 0.28.0.2 > Marathon: 0.15.3-1.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-5770) Mesos state api reporting Host IP instead of container IP with health check
[ https://issues.apache.org/jira/browse/MESOS-5770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362755#comment-15362755 ] Lax edited comment on MESOS-5770 at 7/5/16 4:58 PM: Is the work around suggested in other ticket good to be picked up? If so, is there a way to build packaged version of mesos? Appreciate if you can share the packaging instructions for Mesos. would like to package mesos with the patch and verify. was (Author: lax77): Is the work around suggested in other ticket good to be picked up? If so, is there a way to build packaged version of mesos? would like to package mesos and try out the patch. > Mesos state api reporting Host IP instead of container IP with health check > > > Key: MESOS-5770 > URL: https://issues.apache.org/jira/browse/MESOS-5770 > Project: Mesos > Issue Type: Bug >Reporter: Lax >Priority: Critical > Fix For: 0.28.0 > > > Am using Mesos IP per container using docker containerizer (via Calico). > Mesos state API (/master/state.json) seems to report container IP as long as > I have no health check on my task. As soon as I add health check to the task, > mesos start reporting Host IP instead of Container IP. > Had initially opened this bug on Marathon > (https://github.com/mesosphere/marathon/issues/3907). But then told the issue > is with Mesos reporting the wrong IP. > Here are versions of Mesos and Marathon I was using. > Mesos: 0.28.0.2 > Marathon: 0.15.3-1.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5770) Mesos state api reporting Host IP instead of container IP with health check
[ https://issues.apache.org/jira/browse/MESOS-5770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362755#comment-15362755 ] Lax commented on MESOS-5770: Is the work around suggested in other ticket good to be picked up? If so, is there a way to build packaged version of mesos? would like to package mesos and try out the patch. > Mesos state api reporting Host IP instead of container IP with health check > > > Key: MESOS-5770 > URL: https://issues.apache.org/jira/browse/MESOS-5770 > Project: Mesos > Issue Type: Bug >Reporter: Lax >Priority: Critical > Fix For: 0.28.0 > > > Am using Mesos IP per container using docker containerizer (via Calico). > Mesos state API (/master/state.json) seems to report container IP as long as > I have no health check on my task. As soon as I add health check to the task, > mesos start reporting Host IP instead of Container IP. > Had initially opened this bug on Marathon > (https://github.com/mesosphere/marathon/issues/3907). But then told the issue > is with Mesos reporting the wrong IP. > Here are versions of Mesos and Marathon I was using. > Mesos: 0.28.0.2 > Marathon: 0.15.3-1.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-5770) Mesos state api reporting Host IP instead of container IP with health check
Lax created MESOS-5770: -- Summary: Mesos state api reporting Host IP instead of container IP with health check Key: MESOS-5770 URL: https://issues.apache.org/jira/browse/MESOS-5770 Project: Mesos Issue Type: Bug Reporter: Lax Priority: Critical Fix For: 0.28.0 Am using Mesos IP per container using docker containerizer (via Calico). Mesos state API (/master/state.json) seems to report container IP as long as I have no health check on my task. As soon as I add health check to the task, mesos start reporting Host IP instead of Container IP. Had initially opened this bug on Marathon (https://github.com/mesosphere/marathon/issues/3907). But then told the issue is with Mesos reporting the wrong IP. Here are versions of Mesos and Marathon I was using. Mesos: 0.28.0.2 Marathon: 0.15.3-1.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-5080) With NetworkManager off, can't seem to deploy docker containerized apps
[ https://issues.apache.org/jira/browse/MESOS-5080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15233230#comment-15233230 ] Lax commented on MESOS-5080: I solved the issue. Since I did not have permissions add comments/edit left the ticket open. Issue was my marathon App definition happened to have Overlay network specific settings and mesos on seeing those settings deployed my app with NONE network mode instead of HOST networking causing Docker not to copy in any of the Network config files along. Once I got rid off those attributes from my definition my container got deployed fine. Thanks for looking into the issue and your time Lax > With NetworkManager off, can't seem to deploy docker containerized apps > --- > > Key: MESOS-5080 > URL: https://issues.apache.org/jira/browse/MESOS-5080 > Project: Mesos > Issue Type: Bug > Components: docker >Affects Versions: 0.27.0 > Environment: Centos 7.1 >Reporter: Lax > > Mesos is unable to deploy docker container apps after switching > NetworkManager service off. > > Noticed none of the Host network config files (like hosts, hostname, > resolv.conf, etc) were seen under the docker container dir > (/var/lib/docker/containers/[container id]) if I provision the container thru > mesos. Only files seen were config.json and hostconfig.json files. > However when I launch same container direct via docker run the issue is not > seen. Container dir has all the network config files. > Any reason why mesos unable to push those to the container dir? is there > specific network setting needs enabled now that I have NetworkManager > switched off? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-5080) With NetworkManager off, can't seem to deploy docker containerized apps
Lax created MESOS-5080: -- Summary: With NetworkManager off, can't seem to deploy docker containerized apps Key: MESOS-5080 URL: https://issues.apache.org/jira/browse/MESOS-5080 Project: Mesos Issue Type: Bug Components: docker Affects Versions: 0.27.0 Environment: Centos 7.1 Reporter: Lax Mesos is unable to deploy docker container apps after switching NetworkManager service off. Noticed none of the Host network config files (like hosts, hostname, resolv.conf, etc) were seen under the docker container dir (/var/lib/docker/containers/[container id]) if I provision the container thru mesos. Only files seen were config.json and hostconfig.json files. However when I launch same container direct via docker run the issue is not seen. Container dir has all the network config files. Any reason why mesos unable to push those to the container dir? is there specific network setting needs enabled now that I have NetworkManager switched off? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
