Re: Regarding Public slave and marathon-lb

[Dhiraj Thakur]

Thanks David,
Instruction on that blog worked for me.
-Dhiraj

From: David Greenberg >
Reply-To: "user@mesos.apache.org" 
>
Date: Monday, 2 May 2016 7:06 pm
To: "user@mesos.apache.org" 
>
Subject: Re: Regarding Public slave and marathon-lb

I found this blog: 
https://mesosphere.com/blog/2016/04/26/mesosphere-guide-getting-started-dcos-part-2/

Go down to the section "adding an internet facing load balancer"--it seems to 
explain how to make a public slave with DC/OS on AWS.

On Mon, May 2, 2016 at 8:10 AM Dhiraj Thakur 
> wrote:
Hi I have done the setup of dcos  by following doc here
https://dcos.io/docs/1.7/administration/installing/custom/cli/

I can't find any info on how can we mark any slave as public slave?
Right now I have 7 slave deployed on cluster out of that 2 slave are on 
internet subnet and dcos is able to detect all the slave.
When I try to install marathon-lb, deployment is getting stuck at waiting state.
Its not providing any info why its in waiting state.

Can some one please help me with mararthon-lb deployment?

Note: I am doing setup on aws.
-Dhiraj

Re: How to use a complete host

[Sharma Podila]

This can't be achieved with the offer model as it stands today, unless you
have only a single framework in the cluster. There is no visibility into
what other resources are available on the agent which weren't offered to
your framework.

However, for the short term, you can use a hack to put in custom attributes
that indicate the total amount of resources available on the agent. So,
when you get the offer, you can verify if you got the offer for the entire
agent's resources.

Mesos-4138 is interesting, as Jeff pointed out.





On Mon, May 2, 2016 at 11:07 AM, haosdent  wrote:

> It sounds like you could reserve resources by a role in that machine. And
> then your framework launched by that role.
>
> On Tue, May 3, 2016 at 1:57 AM, Christoph Heer 
> wrote:
>
>> Hi everyone,
>>
>> sometimes in my Mesos use-case it's required to ensure that my own
>> framework is able to schedule a task which consume all resources of a
>> machine.
>>
>> Do you have some advises how to implement such a scheduler. Is there
>> another scheduler which already implemented something similar?
>>
>> Thank you and best regards
>> Christoph
>>
>
>
>
> --
> Best Regards,
> Haosdent Huang
>

Re: How to use a complete host

[Jeff Schroeder]

In addition to actually using roles, you might also consider subscribing to
MESOS-4138, which will implement this idea in a generic way (but won't help
you today).

On Monday, May 2, 2016, Christoph Heer  wrote:

> Hi everyone,
>
> sometimes in my Mesos use-case it's required to ensure that my own
> framework is able to schedule a task which consume all resources of a
> machine.
>
> Do you have some advises how to implement such a scheduler. Is there
> another scheduler which already implemented something similar?
>
> Thank you and best regards
> Christoph
>


-- 
Text by Jeff, typos by iPhone

Status of MESOS-2533?

[Jeff Schroeder]

Some frameworks like Aurora use custom executors to distribute the
healthchecks with the tasks. This allows the task to survive a network
partition without the scheduler setting it to TASK_LOST.

Marathon uses mesos-health-check for command based health checks, but does
TCP and HTTP healthchecks from the elected scheduler (marathon issue
#3728). On a partition event, it sets those tasks to TASK_LOST causing the
master to kill them on partition heal. It also means the scheduler gets
bogged down when you have many tasks with many healthchecks defined.

Can this feature get a Shepard as would be useful for making mesos tasks
more resilient in general? There is an open review from Haosdent for fixing
it.

Thanks!


-- 
Text by Jeff, typos by iPhone

Re: Passing command trough configuration json

[Evan Krall]

Your command /usr/sbin/sshd -D && mkdir /tmp/test will run sshd in the
foreground, wait until it finishes, and then run mkdir.

Perhaps you want to put mkdir before sshd, since mkdir should complete
quickly.

On Mon, May 2, 2016 at 1:13 PM, Rayees Namathponnan 
wrote:

> Hi All,
>
> In Docker file i have entry point  ENTRYPOINT ["/usr/sbin/sshd", "-D"]
> I want to run run some command (eg : mkdir /tmp/test ) while staring the 
> container, since you add command in
> configuration your default entry point wont work, so I added command like 
> "/usr/sbin/sshd -D && mkdir /tmp/test "
> with the expectation both ssh start and mkdir will be executed while strating 
> the contanier.
>
> Unfortunately mkdir command not executing but ssh works fine, not sure 
> command i provided worked itself
> or container started with entry point provided in docker file
>
> Any help ?
>
>
> Regards,
> Rayees
>
>
>

Passing command trough configuration json

[Rayees Namathponnan]

Hi All,

In Docker file i have entry point  ENTRYPOINT ["/usr/sbin/sshd", "-D"]
I want to run run some command (eg : mkdir /tmp/test ) while staring
the container, since you add command in
configuration your default entry point wont work, so I added command
like "/usr/sbin/sshd -D && mkdir /tmp/test "
with the expectation both ssh start and mkdir will be executed while
strating the contanier.

Unfortunately mkdir command not executing but ssh works fine, not sure
command i provided worked itself
or container started with entry point provided in docker file

Any help ?


Regards,
Rayees

Re: How to install marathon-lb

[Kamil Wokitajtis]

Did you try to run docker command from shell? Does it work for you?
Do you have also slave running on 131.154.96.27?
In my case entry was "constraints": [["hostname", "CLUSTER", "master-1"]],
because on host master-1 I have both master and slave running.
If you don't have slave running on 131.154.96.27, update this entry to
point to one of your slave hosts.


2016-05-02 19:03 GMT+02:00 Stefano Bianchi :

> Here it is:
>
> {
>
>   "id": "marathon-lb",
>
>   "volumes": [],
>
>   "cpus": 1,
>
>   "mem": 512,
>
>   "instances": 1,
>
>   "container": {
>
> "type": "DOCKER",
>
> "docker": {
>
> "image": "mesosphere/marathon-lb",
>
> "network": "HOST",
>
> "parameters": [
>
> { "key": "env", "value": "PORTS=9090" }
>
> ],
>
> "portMappings": [
>
>   {
>
>"containerPort": 80,
>
> "hostPort": 80,
>
> "servicePort": 10004,
>
> "protocol": "tcp"
>
>   },
>
>   {
>
>"containerPort": 9090,
>
> "hostPort": 9090,
>
> "servicePort": 10005,
>
> "protocol": "tcp"
>
>   }
>
> ]},
>
> "privileged": false
>
>   },
>
>   "args": [
>
>   "sse",
>
>   "--marathon", "http://192.168.100.54:8080;,
>
>   "--group", "external"
>
>   ],
>
>   "forcePullImage": true,
>
>   "constraints": [["hostname", "CLUSTER", "131.154.96.27"]]
>
> }
>
>
> where:
> 192.168.100.54 is the private IP there marathon is running.
>
> 131.154.96.27: is the hostname i set in /etc/mesos-master/hostname and
> /etc/mesos-master/ip files.
>
>
>
> 2016-05-02 17:20 GMT+02:00 Kamil Wokitajtis :
>
>> Please paste your modified marathon-lb.json file.
>> Can you see any error messages in the marathon and mesos logs ?
>> We are mapping ports 80, 9090,10004,10005, those need to be open.
>>
>> Docker command is as follows:
>>
>> docker run --privileged -e PORTS=9090 --net=host -p 80:80 -p 9090:9090
>> mesosphere/marathon-lb sse --marathon http://192.168.33.20:8080 --group
>> "external"
>>
>> 2016-05-02 17:07 GMT+02:00 Stefano Bianchi :
>>
>>> Can you please tell me which is the correspondent marathon-lb.json
>>> docker command?
>>>
>>> 2016-05-02 16:55 GMT+02:00 Stefano Bianchi :
>>>
 All those ports must be open? Because i am on openstack environment and
 i need to know if there is something to open

 2016-05-02 15:52 GMT+02:00 Stefano Bianchi :

> i changed all but i see waiting app.
>
> 2016-05-02 15:26 GMT+02:00 Kamil Wokitajtis :
>
>> I am using private docker registry (https://docs.docker.com/registry/
>> ).
>> In my case I can pull image from local repository installed on node
>> master-1 (master-1:5000/marathon-lb):
>>  "image": "master-1:5000/marathon-lb"
>>
>> If you don't have private registry, you have to pull image from the
>> docker hub (mesosphere/marathon-lb):
>>  "image": "mesosphere/marathon-lb"
>>
>> Basically, for many reasons I want all my images to be hosted locally.
>>
>>
>>
>> 2016-05-02 15:10 GMT+02:00 Stefano Bianchi :
>>
>>> Thanks your your answer.
>>> I understood what you are telling me, just a thing, what you mean
>>> with docker registry?
>>>
>>> 2016-05-02 13:31 GMT+02:00 Kamil Wokitajtis :
>>>
 Hi,

 Fastest way is to run marathon-lb docker image.
 You have to update marathon-lb.json code below with yours (or
 public) docker registry, IPs and hostnames.
 When you add application to marathon and you want it to be load
 balanced via marathon-lb,
 you have to add labels section eg.:

  "labels": {
 "HAPROXY_GROUP":"external",
 "HAPROXY_0_VHOST":"service.mesosphere.com"
   }


 Reading documentation on github was enough for me, specially
 "issues" tab ;)
 https://github.com/mesosphere/marathon-lb


 Below code works for me, but I think you can change "network" mode
 from BRIDGE to HOST and remove line : { "key": "net", "value": "host" 
 }.

 marathon-lb.json:

 {
   "id": "marathon-lb",
   "volumes": [],
   "cpus": 0.5,
   "mem": 256.0,
   "instances": 1,
   "container": {
 "type": "DOCKER",
 "docker": {
 "image": "master-1:5000/marathon-lb",
 "network": "BRIDGE",
 "parameters": [
 { "key": "env", "value": "PORTS=9090" },
 { "key": "net", "value": "host" }
 ],
 "portMappings": [
   {
 "containerPort": 80,
 "hostPort": 80,
 "servicePort": 10004,

Re: How to use a complete host

[haosdent]

It sounds like you could reserve resources by a role in that machine. And
then your framework launched by that role.

On Tue, May 3, 2016 at 1:57 AM, Christoph Heer 
wrote:

> Hi everyone,
>
> sometimes in my Mesos use-case it's required to ensure that my own
> framework is able to schedule a task which consume all resources of a
> machine.
>
> Do you have some advises how to implement such a scheduler. Is there
> another scheduler which already implemented something similar?
>
> Thank you and best regards
> Christoph
>



-- 
Best Regards,
Haosdent Huang

How to use a complete host

[Christoph Heer]

Hi everyone,

sometimes in my Mesos use-case it's required to ensure that my own framework is 
able to schedule a task which consume all resources of a machine. 

Do you have some advises how to implement such a scheduler. Is there another 
scheduler which already implemented something similar?

Thank you and best regards
Christoph 

Re: Offers to a framework

[Joseph Wu]

Both the Mesos master and Marathon have metrics that tell you how many
offers have been sent, but not the contents of said offers.  Marathon does
not keep offers long enough for them to show up as "outstanding offers" in
the Mesos UI.

As far as I know, one way to get the offer contents is by setting logging
verbosity to GLOG_v=2 (*Warning*: This prints a lot of stuff).  When the
master is started or toggled to this logging level, the allocator will
print each offer before it sends it:
https://github.com/apache/mesos/blob/2c6eeefe13c5706da70328af6cea14f802121bf2/src/master/allocator/mesos/hierarchical.cpp#L1462-L1463

For how to temporarily toggle the logging verbosity:
http://mesos.apache.org/documentation/latest/logging/

On Mon, May 2, 2016 at 6:46 AM, Vaibhav Khanduja 
wrote:

> For a use-case, we need to know offers been sent to a registered
> framework. For e.g. if we are using marathon as a framework, the master
> would send offers to it based on DRF as soon as a slave is available. The
> marathon would then accept the offer if it requires to start a job. Is
> there is way to know externally using an api - either master or marathon or
> framework to know what offers are been sent and what are been accepted?
>
> Thx
>

Re: Offers to a framework

[haosdent]

If you could modify the framework code, you could inject this login in
`resourceOffers`.

On Mon, May 2, 2016 at 9:46 PM, Vaibhav Khanduja 
wrote:

> For a use-case, we need to know offers been sent to a registered
> framework. For e.g. if we are using marathon as a framework, the master
> would send offers to it based on DRF as soon as a slave is available. The
> marathon would then accept the offer if it requires to start a job. Is
> there is way to know externally using an api - either master or marathon or
> framework to know what offers are been sent and what are been accepted?
>
> Thx
>



-- 
Best Regards,
Haosdent Huang

Re: How to install marathon-lb

[Stefano Bianchi]

Here it is:

{

  "id": "marathon-lb",

  "volumes": [],

  "cpus": 1,

  "mem": 512,

  "instances": 1,

  "container": {

"type": "DOCKER",

"docker": {

"image": "mesosphere/marathon-lb",

"network": "HOST",

"parameters": [

{ "key": "env", "value": "PORTS=9090" }

],

"portMappings": [

  {

   "containerPort": 80,

"hostPort": 80,

"servicePort": 10004,

"protocol": "tcp"

  },

  {

   "containerPort": 9090,

"hostPort": 9090,

"servicePort": 10005,

"protocol": "tcp"

  }

]},

"privileged": false

  },

  "args": [

  "sse",

  "--marathon", "http://192.168.100.54:8080;,

  "--group", "external"

  ],

  "forcePullImage": true,

  "constraints": [["hostname", "CLUSTER", "131.154.96.27"]]

}


where:
192.168.100.54 is the private IP there marathon is running.

131.154.96.27: is the hostname i set in /etc/mesos-master/hostname and
/etc/mesos-master/ip files.



2016-05-02 17:20 GMT+02:00 Kamil Wokitajtis :

> Please paste your modified marathon-lb.json file.
> Can you see any error messages in the marathon and mesos logs ?
> We are mapping ports 80, 9090,10004,10005, those need to be open.
>
> Docker command is as follows:
>
> docker run --privileged -e PORTS=9090 --net=host -p 80:80 -p 9090:9090
> mesosphere/marathon-lb sse --marathon http://192.168.33.20:8080 --group
> "external"
>
> 2016-05-02 17:07 GMT+02:00 Stefano Bianchi :
>
>> Can you please tell me which is the correspondent marathon-lb.json docker
>> command?
>>
>> 2016-05-02 16:55 GMT+02:00 Stefano Bianchi :
>>
>>> All those ports must be open? Because i am on openstack environment and
>>> i need to know if there is something to open
>>>
>>> 2016-05-02 15:52 GMT+02:00 Stefano Bianchi :
>>>
 i changed all but i see waiting app.

 2016-05-02 15:26 GMT+02:00 Kamil Wokitajtis :

> I am using private docker registry (https://docs.docker.com/registry/
> ).
> In my case I can pull image from local repository installed on node
> master-1 (master-1:5000/marathon-lb):
>  "image": "master-1:5000/marathon-lb"
>
> If you don't have private registry, you have to pull image from the
> docker hub (mesosphere/marathon-lb):
>  "image": "mesosphere/marathon-lb"
>
> Basically, for many reasons I want all my images to be hosted locally.
>
>
>
> 2016-05-02 15:10 GMT+02:00 Stefano Bianchi :
>
>> Thanks your your answer.
>> I understood what you are telling me, just a thing, what you mean
>> with docker registry?
>>
>> 2016-05-02 13:31 GMT+02:00 Kamil Wokitajtis :
>>
>>> Hi,
>>>
>>> Fastest way is to run marathon-lb docker image.
>>> You have to update marathon-lb.json code below with yours (or
>>> public) docker registry, IPs and hostnames.
>>> When you add application to marathon and you want it to be load
>>> balanced via marathon-lb,
>>> you have to add labels section eg.:
>>>
>>>  "labels": {
>>> "HAPROXY_GROUP":"external",
>>> "HAPROXY_0_VHOST":"service.mesosphere.com"
>>>   }
>>>
>>>
>>> Reading documentation on github was enough for me, specially
>>> "issues" tab ;)
>>> https://github.com/mesosphere/marathon-lb
>>>
>>>
>>> Below code works for me, but I think you can change "network" mode
>>> from BRIDGE to HOST and remove line : { "key": "net", "value": "host" }.
>>>
>>> marathon-lb.json:
>>>
>>> {
>>>   "id": "marathon-lb",
>>>   "volumes": [],
>>>   "cpus": 0.5,
>>>   "mem": 256.0,
>>>   "instances": 1,
>>>   "container": {
>>> "type": "DOCKER",
>>> "docker": {
>>> "image": "master-1:5000/marathon-lb",
>>> "network": "BRIDGE",
>>> "parameters": [
>>> { "key": "env", "value": "PORTS=9090" },
>>> { "key": "net", "value": "host" }
>>> ],
>>> "portMappings": [
>>>   {
>>> "containerPort": 80,
>>> "hostPort": 80,
>>> "servicePort": 10004,
>>> "protocol": "tcp"
>>>   },
>>>   {
>>> "containerPort": 9090,
>>> "hostPort": 9090,
>>> "servicePort": 10005,
>>> "protocol": "tcp"
>>>   }
>>> ]},
>>> "privileged": false
>>>   },
>>>   "args": [
>>>   "sse",
>>>   "--marathon", "http://192.168.33.20:8080;,
>>>   "--group", "external"
>>>   ],
>>>   "forcePullImage": true,
>>>   "constraints": [["hostname", "CLUSTER", "master-1"]]
>>> }
>>>
>>> 2016-05-02 11:08 GMT+02:00 Stefano Bianchi :
>>>
 Hello all

 

Re: How to install marathon-lb

[Kamil Wokitajtis]

Please paste your modified marathon-lb.json file.
Can you see any error messages in the marathon and mesos logs ?
We are mapping ports 80, 9090,10004,10005, those need to be open.

Docker command is as follows:

docker run --privileged -e PORTS=9090 --net=host -p 80:80 -p 9090:9090
mesosphere/marathon-lb sse --marathon http://192.168.33.20:8080 --group
"external"

2016-05-02 17:07 GMT+02:00 Stefano Bianchi :

> Can you please tell me which is the correspondent marathon-lb.json docker
> command?
>
> 2016-05-02 16:55 GMT+02:00 Stefano Bianchi :
>
>> All those ports must be open? Because i am on openstack environment and i
>> need to know if there is something to open
>>
>> 2016-05-02 15:52 GMT+02:00 Stefano Bianchi :
>>
>>> i changed all but i see waiting app.
>>>
>>> 2016-05-02 15:26 GMT+02:00 Kamil Wokitajtis :
>>>
 I am using private docker registry (https://docs.docker.com/registry/
 ).
 In my case I can pull image from local repository installed on node
 master-1 (master-1:5000/marathon-lb):
  "image": "master-1:5000/marathon-lb"

 If you don't have private registry, you have to pull image from the
 docker hub (mesosphere/marathon-lb):
  "image": "mesosphere/marathon-lb"

 Basically, for many reasons I want all my images to be hosted locally.



 2016-05-02 15:10 GMT+02:00 Stefano Bianchi :

> Thanks your your answer.
> I understood what you are telling me, just a thing, what you mean with
> docker registry?
>
> 2016-05-02 13:31 GMT+02:00 Kamil Wokitajtis :
>
>> Hi,
>>
>> Fastest way is to run marathon-lb docker image.
>> You have to update marathon-lb.json code below with yours (or public)
>> docker registry, IPs and hostnames.
>> When you add application to marathon and you want it to be load
>> balanced via marathon-lb,
>> you have to add labels section eg.:
>>
>>  "labels": {
>> "HAPROXY_GROUP":"external",
>> "HAPROXY_0_VHOST":"service.mesosphere.com"
>>   }
>>
>>
>> Reading documentation on github was enough for me, specially "issues"
>> tab ;)
>> https://github.com/mesosphere/marathon-lb
>>
>>
>> Below code works for me, but I think you can change "network" mode
>> from BRIDGE to HOST and remove line : { "key": "net", "value": "host" }.
>>
>> marathon-lb.json:
>>
>> {
>>   "id": "marathon-lb",
>>   "volumes": [],
>>   "cpus": 0.5,
>>   "mem": 256.0,
>>   "instances": 1,
>>   "container": {
>> "type": "DOCKER",
>> "docker": {
>> "image": "master-1:5000/marathon-lb",
>> "network": "BRIDGE",
>> "parameters": [
>> { "key": "env", "value": "PORTS=9090" },
>> { "key": "net", "value": "host" }
>> ],
>> "portMappings": [
>>   {
>> "containerPort": 80,
>> "hostPort": 80,
>> "servicePort": 10004,
>> "protocol": "tcp"
>>   },
>>   {
>> "containerPort": 9090,
>> "hostPort": 9090,
>> "servicePort": 10005,
>> "protocol": "tcp"
>>   }
>> ]},
>> "privileged": false
>>   },
>>   "args": [
>>   "sse",
>>   "--marathon", "http://192.168.33.20:8080;,
>>   "--group", "external"
>>   ],
>>   "forcePullImage": true,
>>   "constraints": [["hostname", "CLUSTER", "master-1"]]
>> }
>>
>> 2016-05-02 11:08 GMT+02:00 Stefano Bianchi :
>>
>>> Hello all
>>>
>>> I'm not using DC/OS, and i would like to know how to install
>>> marathon-lb on my mesos cluster. Is there any tutorial that does not
>>> involve dcos instructions?
>>>
>>> Thanks for replies.
>>>
>>
>>
>

>>>
>>
>

Re: How to install marathon-lb

[Stefano Bianchi]

Can you please tell me which is the correspondent marathon-lb.json docker
command?

2016-05-02 16:55 GMT+02:00 Stefano Bianchi :

> All those ports must be open? Because i am on openstack environment and i
> need to know if there is something to open
>
> 2016-05-02 15:52 GMT+02:00 Stefano Bianchi :
>
>> i changed all but i see waiting app.
>>
>> 2016-05-02 15:26 GMT+02:00 Kamil Wokitajtis :
>>
>>> I am using private docker registry (https://docs.docker.com/registry/).
>>> In my case I can pull image from local repository installed on node
>>> master-1 (master-1:5000/marathon-lb):
>>>  "image": "master-1:5000/marathon-lb"
>>>
>>> If you don't have private registry, you have to pull image from the
>>> docker hub (mesosphere/marathon-lb):
>>>  "image": "mesosphere/marathon-lb"
>>>
>>> Basically, for many reasons I want all my images to be hosted locally.
>>>
>>>
>>>
>>> 2016-05-02 15:10 GMT+02:00 Stefano Bianchi :
>>>
 Thanks your your answer.
 I understood what you are telling me, just a thing, what you mean with
 docker registry?

 2016-05-02 13:31 GMT+02:00 Kamil Wokitajtis :

> Hi,
>
> Fastest way is to run marathon-lb docker image.
> You have to update marathon-lb.json code below with yours (or public)
> docker registry, IPs and hostnames.
> When you add application to marathon and you want it to be load
> balanced via marathon-lb,
> you have to add labels section eg.:
>
>  "labels": {
> "HAPROXY_GROUP":"external",
> "HAPROXY_0_VHOST":"service.mesosphere.com"
>   }
>
>
> Reading documentation on github was enough for me, specially "issues"
> tab ;)
> https://github.com/mesosphere/marathon-lb
>
>
> Below code works for me, but I think you can change "network" mode
> from BRIDGE to HOST and remove line : { "key": "net", "value": "host" }.
>
> marathon-lb.json:
>
> {
>   "id": "marathon-lb",
>   "volumes": [],
>   "cpus": 0.5,
>   "mem": 256.0,
>   "instances": 1,
>   "container": {
> "type": "DOCKER",
> "docker": {
> "image": "master-1:5000/marathon-lb",
> "network": "BRIDGE",
> "parameters": [
> { "key": "env", "value": "PORTS=9090" },
> { "key": "net", "value": "host" }
> ],
> "portMappings": [
>   {
> "containerPort": 80,
> "hostPort": 80,
> "servicePort": 10004,
> "protocol": "tcp"
>   },
>   {
> "containerPort": 9090,
> "hostPort": 9090,
> "servicePort": 10005,
> "protocol": "tcp"
>   }
> ]},
> "privileged": false
>   },
>   "args": [
>   "sse",
>   "--marathon", "http://192.168.33.20:8080;,
>   "--group", "external"
>   ],
>   "forcePullImage": true,
>   "constraints": [["hostname", "CLUSTER", "master-1"]]
> }
>
> 2016-05-02 11:08 GMT+02:00 Stefano Bianchi :
>
>> Hello all
>>
>> I'm not using DC/OS, and i would like to know how to install
>> marathon-lb on my mesos cluster. Is there any tutorial that does not
>> involve dcos instructions?
>>
>> Thanks for replies.
>>
>
>

>>>
>>
>

Re: How to install marathon-lb

[Stefano Bianchi]

All those ports must be open? Because i am on openstack environment and i
need to know if there is something to open

2016-05-02 15:52 GMT+02:00 Stefano Bianchi :

> i changed all but i see waiting app.
>
> 2016-05-02 15:26 GMT+02:00 Kamil Wokitajtis :
>
>> I am using private docker registry (https://docs.docker.com/registry/).
>> In my case I can pull image from local repository installed on node
>> master-1 (master-1:5000/marathon-lb):
>>  "image": "master-1:5000/marathon-lb"
>>
>> If you don't have private registry, you have to pull image from the
>> docker hub (mesosphere/marathon-lb):
>>  "image": "mesosphere/marathon-lb"
>>
>> Basically, for many reasons I want all my images to be hosted locally.
>>
>>
>>
>> 2016-05-02 15:10 GMT+02:00 Stefano Bianchi :
>>
>>> Thanks your your answer.
>>> I understood what you are telling me, just a thing, what you mean with
>>> docker registry?
>>>
>>> 2016-05-02 13:31 GMT+02:00 Kamil Wokitajtis :
>>>
 Hi,

 Fastest way is to run marathon-lb docker image.
 You have to update marathon-lb.json code below with yours (or public)
 docker registry, IPs and hostnames.
 When you add application to marathon and you want it to be load
 balanced via marathon-lb,
 you have to add labels section eg.:

  "labels": {
 "HAPROXY_GROUP":"external",
 "HAPROXY_0_VHOST":"service.mesosphere.com"
   }


 Reading documentation on github was enough for me, specially "issues"
 tab ;)
 https://github.com/mesosphere/marathon-lb


 Below code works for me, but I think you can change "network" mode from
 BRIDGE to HOST and remove line : { "key": "net", "value": "host" }.

 marathon-lb.json:

 {
   "id": "marathon-lb",
   "volumes": [],
   "cpus": 0.5,
   "mem": 256.0,
   "instances": 1,
   "container": {
 "type": "DOCKER",
 "docker": {
 "image": "master-1:5000/marathon-lb",
 "network": "BRIDGE",
 "parameters": [
 { "key": "env", "value": "PORTS=9090" },
 { "key": "net", "value": "host" }
 ],
 "portMappings": [
   {
 "containerPort": 80,
 "hostPort": 80,
 "servicePort": 10004,
 "protocol": "tcp"
   },
   {
 "containerPort": 9090,
 "hostPort": 9090,
 "servicePort": 10005,
 "protocol": "tcp"
   }
 ]},
 "privileged": false
   },
   "args": [
   "sse",
   "--marathon", "http://192.168.33.20:8080;,
   "--group", "external"
   ],
   "forcePullImage": true,
   "constraints": [["hostname", "CLUSTER", "master-1"]]
 }

 2016-05-02 11:08 GMT+02:00 Stefano Bianchi :

> Hello all
>
> I'm not using DC/OS, and i would like to know how to install
> marathon-lb on my mesos cluster. Is there any tutorial that does not
> involve dcos instructions?
>
> Thanks for replies.
>


>>>
>>
>

Re: How to install marathon-lb

[Stefano Bianchi]

i changed all but i see waiting app.

2016-05-02 15:26 GMT+02:00 Kamil Wokitajtis :

> I am using private docker registry (https://docs.docker.com/registry/).
> In my case I can pull image from local repository installed on node
> master-1 (master-1:5000/marathon-lb):
>  "image": "master-1:5000/marathon-lb"
>
> If you don't have private registry, you have to pull image from the docker
> hub (mesosphere/marathon-lb):
>  "image": "mesosphere/marathon-lb"
>
> Basically, for many reasons I want all my images to be hosted locally.
>
>
>
> 2016-05-02 15:10 GMT+02:00 Stefano Bianchi :
>
>> Thanks your your answer.
>> I understood what you are telling me, just a thing, what you mean with
>> docker registry?
>>
>> 2016-05-02 13:31 GMT+02:00 Kamil Wokitajtis :
>>
>>> Hi,
>>>
>>> Fastest way is to run marathon-lb docker image.
>>> You have to update marathon-lb.json code below with yours (or public)
>>> docker registry, IPs and hostnames.
>>> When you add application to marathon and you want it to be load balanced
>>> via marathon-lb,
>>> you have to add labels section eg.:
>>>
>>>  "labels": {
>>> "HAPROXY_GROUP":"external",
>>> "HAPROXY_0_VHOST":"service.mesosphere.com"
>>>   }
>>>
>>>
>>> Reading documentation on github was enough for me, specially "issues"
>>> tab ;)
>>> https://github.com/mesosphere/marathon-lb
>>>
>>>
>>> Below code works for me, but I think you can change "network" mode from
>>> BRIDGE to HOST and remove line : { "key": "net", "value": "host" }.
>>>
>>> marathon-lb.json:
>>>
>>> {
>>>   "id": "marathon-lb",
>>>   "volumes": [],
>>>   "cpus": 0.5,
>>>   "mem": 256.0,
>>>   "instances": 1,
>>>   "container": {
>>> "type": "DOCKER",
>>> "docker": {
>>> "image": "master-1:5000/marathon-lb",
>>> "network": "BRIDGE",
>>> "parameters": [
>>> { "key": "env", "value": "PORTS=9090" },
>>> { "key": "net", "value": "host" }
>>> ],
>>> "portMappings": [
>>>   {
>>> "containerPort": 80,
>>> "hostPort": 80,
>>> "servicePort": 10004,
>>> "protocol": "tcp"
>>>   },
>>>   {
>>> "containerPort": 9090,
>>> "hostPort": 9090,
>>> "servicePort": 10005,
>>> "protocol": "tcp"
>>>   }
>>> ]},
>>> "privileged": false
>>>   },
>>>   "args": [
>>>   "sse",
>>>   "--marathon", "http://192.168.33.20:8080;,
>>>   "--group", "external"
>>>   ],
>>>   "forcePullImage": true,
>>>   "constraints": [["hostname", "CLUSTER", "master-1"]]
>>> }
>>>
>>> 2016-05-02 11:08 GMT+02:00 Stefano Bianchi :
>>>
 Hello all

 I'm not using DC/OS, and i would like to know how to install
 marathon-lb on my mesos cluster. Is there any tutorial that does not
 involve dcos instructions?

 Thanks for replies.

>>>
>>>
>>
>

Offers to a framework

[Vaibhav Khanduja]

For a use-case, we need to know offers been sent to a registered framework.
For e.g. if we are using marathon as a framework, the master would send
offers to it based on DRF as soon as a slave is available. The marathon
would then accept the offer if it requires to start a job. Is there is way
to know externally using an api - either master or marathon or framework to
know what offers are been sent and what are been accepted?

Thx

Re: How to install marathon-lb

[Kamil Wokitajtis]

I am using private docker registry (https://docs.docker.com/registry/).
In my case I can pull image from local repository installed on node
master-1 (master-1:5000/marathon-lb):
 "image": "master-1:5000/marathon-lb"

If you don't have private registry, you have to pull image from the docker
hub (mesosphere/marathon-lb):
 "image": "mesosphere/marathon-lb"

Basically, for many reasons I want all my images to be hosted locally.



2016-05-02 15:10 GMT+02:00 Stefano Bianchi :

> Thanks your your answer.
> I understood what you are telling me, just a thing, what you mean with
> docker registry?
>
> 2016-05-02 13:31 GMT+02:00 Kamil Wokitajtis :
>
>> Hi,
>>
>> Fastest way is to run marathon-lb docker image.
>> You have to update marathon-lb.json code below with yours (or public)
>> docker registry, IPs and hostnames.
>> When you add application to marathon and you want it to be load balanced
>> via marathon-lb,
>> you have to add labels section eg.:
>>
>>  "labels": {
>> "HAPROXY_GROUP":"external",
>> "HAPROXY_0_VHOST":"service.mesosphere.com"
>>   }
>>
>>
>> Reading documentation on github was enough for me, specially "issues" tab
>> ;)
>> https://github.com/mesosphere/marathon-lb
>>
>>
>> Below code works for me, but I think you can change "network" mode from
>> BRIDGE to HOST and remove line : { "key": "net", "value": "host" }.
>>
>> marathon-lb.json:
>>
>> {
>>   "id": "marathon-lb",
>>   "volumes": [],
>>   "cpus": 0.5,
>>   "mem": 256.0,
>>   "instances": 1,
>>   "container": {
>> "type": "DOCKER",
>> "docker": {
>> "image": "master-1:5000/marathon-lb",
>> "network": "BRIDGE",
>> "parameters": [
>> { "key": "env", "value": "PORTS=9090" },
>> { "key": "net", "value": "host" }
>> ],
>> "portMappings": [
>>   {
>> "containerPort": 80,
>> "hostPort": 80,
>> "servicePort": 10004,
>> "protocol": "tcp"
>>   },
>>   {
>> "containerPort": 9090,
>> "hostPort": 9090,
>> "servicePort": 10005,
>> "protocol": "tcp"
>>   }
>> ]},
>> "privileged": false
>>   },
>>   "args": [
>>   "sse",
>>   "--marathon", "http://192.168.33.20:8080;,
>>   "--group", "external"
>>   ],
>>   "forcePullImage": true,
>>   "constraints": [["hostname", "CLUSTER", "master-1"]]
>> }
>>
>> 2016-05-02 11:08 GMT+02:00 Stefano Bianchi :
>>
>>> Hello all
>>>
>>> I'm not using DC/OS, and i would like to know how to install marathon-lb
>>> on my mesos cluster. Is there any tutorial that does not involve dcos
>>> instructions?
>>>
>>> Thanks for replies.
>>>
>>
>>
>

Re: How to install marathon-lb

[Stefano Bianchi]

Thanks your your answer.
I understood what you are telling me, just a thing, what you mean with
docker registry?

2016-05-02 13:31 GMT+02:00 Kamil Wokitajtis :

> Hi,
>
> Fastest way is to run marathon-lb docker image.
> You have to update marathon-lb.json code below with yours (or public)
> docker registry, IPs and hostnames.
> When you add application to marathon and you want it to be load balanced
> via marathon-lb,
> you have to add labels section eg.:
>
>  "labels": {
> "HAPROXY_GROUP":"external",
> "HAPROXY_0_VHOST":"service.mesosphere.com"
>   }
>
>
> Reading documentation on github was enough for me, specially "issues" tab
> ;)
> https://github.com/mesosphere/marathon-lb
>
>
> Below code works for me, but I think you can change "network" mode from
> BRIDGE to HOST and remove line : { "key": "net", "value": "host" }.
>
> marathon-lb.json:
>
> {
>   "id": "marathon-lb",
>   "volumes": [],
>   "cpus": 0.5,
>   "mem": 256.0,
>   "instances": 1,
>   "container": {
> "type": "DOCKER",
> "docker": {
> "image": "master-1:5000/marathon-lb",
> "network": "BRIDGE",
> "parameters": [
> { "key": "env", "value": "PORTS=9090" },
> { "key": "net", "value": "host" }
> ],
> "portMappings": [
>   {
> "containerPort": 80,
> "hostPort": 80,
> "servicePort": 10004,
> "protocol": "tcp"
>   },
>   {
> "containerPort": 9090,
> "hostPort": 9090,
> "servicePort": 10005,
> "protocol": "tcp"
>   }
> ]},
> "privileged": false
>   },
>   "args": [
>   "sse",
>   "--marathon", "http://192.168.33.20:8080;,
>   "--group", "external"
>   ],
>   "forcePullImage": true,
>   "constraints": [["hostname", "CLUSTER", "master-1"]]
> }
>
> 2016-05-02 11:08 GMT+02:00 Stefano Bianchi :
>
>> Hello all
>>
>> I'm not using DC/OS, and i would like to know how to install marathon-lb
>> on my mesos cluster. Is there any tutorial that does not involve dcos
>> instructions?
>>
>> Thanks for replies.
>>
>
>

Regarding Public slave and marathon-lb

[Dhiraj Thakur]

Hi I have done the setup of dcos  by following doc here
https://dcos.io/docs/1.7/administration/installing/custom/cli/

I can't find any info on how can we mark any slave as public slave?
Right now I have 7 slave deployed on cluster out of that 2 slave are on 
internet subnet and dcos is able to detect all the slave.
When I try to install marathon-lb, deployment is getting stuck at waiting state.
Its not providing any info why its in waiting state.

Can some one please help me with mararthon-lb deployment?

Note: I am doing setup on aws.
-Dhiraj

Re: How to install marathon-lb

[Kamil Wokitajtis]

Hi,

Fastest way is to run marathon-lb docker image.
You have to update marathon-lb.json code below with yours (or public)
docker registry, IPs and hostnames.
When you add application to marathon and you want it to be load balanced
via marathon-lb,
you have to add labels section eg.:

 "labels": {
"HAPROXY_GROUP":"external",
"HAPROXY_0_VHOST":"service.mesosphere.com"
  }


Reading documentation on github was enough for me, specially "issues" tab ;)
https://github.com/mesosphere/marathon-lb


Below code works for me, but I think you can change "network" mode from
BRIDGE to HOST and remove line : { "key": "net", "value": "host" }.

marathon-lb.json:

{
  "id": "marathon-lb",
  "volumes": [],
  "cpus": 0.5,
  "mem": 256.0,
  "instances": 1,
  "container": {
"type": "DOCKER",
"docker": {
"image": "master-1:5000/marathon-lb",
"network": "BRIDGE",
"parameters": [
{ "key": "env", "value": "PORTS=9090" },
{ "key": "net", "value": "host" }
],
"portMappings": [
  {
"containerPort": 80,
"hostPort": 80,
"servicePort": 10004,
"protocol": "tcp"
  },
  {
"containerPort": 9090,
"hostPort": 9090,
"servicePort": 10005,
"protocol": "tcp"
  }
]},
"privileged": false
  },
  "args": [
  "sse",
  "--marathon", "http://192.168.33.20:8080;,
  "--group", "external"
  ],
  "forcePullImage": true,
  "constraints": [["hostname", "CLUSTER", "master-1"]]
}

2016-05-02 11:08 GMT+02:00 Stefano Bianchi :

> Hello all
>
> I'm not using DC/OS, and i would like to know how to install marathon-lb
> on my mesos cluster. Is there any tutorial that does not involve dcos
> instructions?
>
> Thanks for replies.
>

Re: DC/OS dynamic reservation dosen't work

[caogaojin]

thanks @haosdent, here is the explanation 
https://github.com/mesosphere/marathon/issues/3851#issuecomment-216192331 

--
Best Regards,

GaoJin CAO 
ChinaMobile(Suzhou) R
Mail：caogaojin at cmss.chinamobile.com 
> On May 2, 2016, at 12:00, haosdent  wrote:
> 
> Actually "unreserved" here in Marathon means Resources which don't contains 
> ReservationInfo. For static reservation, they didn't contains 
> ReservationInfo, so they are "unreserved". For dynamic reservation, they have 
> ReservationInfo, so they are "reserved". This is why your task works in 
> static reservation while could not work in dynamic reservation.
> 
> On Mon, May 2, 2016 at 9:10 AM, caogaojin  > wrote:
> @Haosdent,  sorry for delay; i’m not familiar with scala, the compilation 
> takes me some time. you are right, after changing the file you told me, it 
> works. But that’s not what i want, it my situation, marathon should takes 
> both reserved resource and unreserved resource; when using official marathon 
> version, marathon will use static reserved resources (mesos agent started 
> with `--default_role dev` flag, and marathon with `--mesos_role dev`), but it 
> seems marathon did not take static reserved resource as  *reserved resource*,
> 
> ```
> Considering unreserved resources with roles {dev}. Not all basic resources 
> satisfied: cpus NOT SATISFIED (1.0 > 0.0), mem NOT SATISFIED (128.0 > 0.0) 
> (mesosphere.mesos.ResourceMatcher$:marathon-akka.actor.default-dispatcher-6)
> ```
> 
> the above shows that marathon only considers *unreserved* resources with role 
> `dev`. However IMO, all resources with role `dev` should be reserved 
> resources, right? i’m quite confused, @haosdent, are resources provided  by 
> mesos agent  that started with `--default_role dev` flag unreseved resources? 
> i will advise mesosphere/marathon community for more help.
> 
> Thanks,
> 
> --
> Best Regards,
> 
> GaoJin CAO 
> ChinaMobile(Suzhou) R
> Mail：caogaojin at cmss.chinamobile.com 
>> On May 1, 2016, at 18:04, haosdent > > wrote:
>> 
>> This may be a feature of Marathon. I saw it only filter resources from those 
>> unreserved resources. After change
>> 
>> ```
>> diff --git a/src/main/scala/mesosphere/mesos/TaskBuilder.scala 
>> b/src/main/scala/mesosphere/mesos/TaskBuilder.scala
>> index efa3406..e750881 100644
>> --- a/src/main/scala/mesosphere/mesos/TaskBuilder.scala
>> +++ b/src/main/scala/mesosphere/mesos/TaskBuilder.scala
>> @@ -80,7 +80,7 @@ class TaskBuilder(app: AppDefinition,
>> 
>>  val resourceMatch =
>>ResourceMatcher.matchResources(
>> -offer, app, runningTasks, ResourceSelector(acceptedResourceRoles, 
>> reserved = false))
>> +offer, app, runningTasks, ResourceSelector(acceptedResourceRoles, 
>> reserved = true))
>> 
>>  build(offer, resourceMatch)
>>}
>> ```
>> 
>> It would use the dynamic resources we reserved before. I find Marathon 
>> document have these words:
>> 
>> ```
>> ### Static Reservations
>> 
>> Dynamic reservations can only be created for unreserved resources. If you 
>> specify an agent's resources to be reserved for a role via the Mesos 
>> `--resources` or `--default_role` flag, these resources cannot be used for 
>> dynamic reservations. In addition, if Marathon is started with the 
>> `--default_accepted_resource_roles` flag specifying a value that does not 
>> contain `*`, your application definition should explicitly specify 
>> `"acceptedResourceRoles": ["*"]` in order to allow usage and reservation of 
>> unreserved cluster resources.
>> ```
>> 
>> You could double check this with Marathon mailing list 
>> https://groups.google.com/forum/?hl=en#!forum/marathon-framework 
>>  . Thanks 
>> a lot!
>> 
>> On Sun, May 1, 2016 at 5:07 PM, haosdent > > wrote:
>> hmm, I am still digging more information in marathon. It may delay because I 
>> am not familiar with marathon's code and sick today, but I would inform you 
>> once have result. My quick conclusion is this may be the bug of marathon 
>> when match resource requirement in offers from Mesos Master. Still under 
>> finding the root cause of it.
>> 
>> On Sun, May 1, 2016 at 4:24 PM, caogaojin > > wrote:
>> @Haosdent, what does http://10.132.46.71:5050/roles 
>>  mean? shouldn’t that tell us all resources 
>> that framework with `dev` role takes, both from `dev` and `*` role? It 
>> doesn’t mean resources are consumed from `dev` role, right?
>> --
>> Best Regards,
>> 
>> GaoJin CAO 
>> ChinaMobile(Suzhou) R
>> Mail：caogaojin at 

How to install marathon-lb

[Stefano Bianchi]

Hello all

I'm not using DC/OS, and i would like to know how to install marathon-lb on
my mesos cluster. Is there any tutorial that does not involve dcos
instructions?

Thanks for replies.

Re: Mesos Web UI

[Adam Bordelon]

Backported to 0.28.x and 0.27.x. The relevant change/typo wasn't introduced
until 0.27.x, so I left 0.26.x alone.

On Fri, Apr 29, 2016 at 9:57 AM, Vinod Kone  wrote:

> Adam, since you committed this, feel free to backport it to the relevant
> stable branches (26.x, 27.x, 28.x). They will be included in the next patch
> releases.
>
> On Fri, Apr 29, 2016 at 9:31 AM, Gilbert Song 
> wrote:
>
>> Julian, since the fix was after 0.28, could you try it again by applying
>> the patch @haosdent provided, or try with mesos master head?
>>
>> On Fri, Apr 29, 2016 at 8:52 AM, Julian Gonzalez Llorente <
>> jgonza...@medallia.com> wrote:
>>
>>> Hello list,
>>>
>>> I found one little error on the web UI of mesos.
>>> In the "Slaves" tab there are a table with the columns: ID, Host, CPUs,
>>> Mem, Disk, Registered and Re-Registered.
>>> But in the "Offers" tab there is two times Mem. : ID, Framework, Host,
>>> Cpus, Mem, Mem.
>>> The second Mem should be Disk instead.
>>>
>>> I suppose that should be easy to fix.
>>>
>>> Regards,
>>> Julian
>>>
>>
>>
>