Re: Attributes cause agent to fail

[Douglas Nelson]

That showed me what wasn't working. If you start the mesos agent at all
before setting attributes (or if you change attributes) you need to make
sure it doesn't recover old live executors.

The error:
Failed to perform recovery: Incompatible agent info detected.

The solution:
Step 1: rm -f /var/lib/mesos/meta/slaves/latest
Step 2: Restart the agent.

Thanks!

On Fri, Jul 29, 2016 at 8:47 PM, Benjamin Mahler  wrote:

> Unfortunately we log termination messages to stderr rather than the
> logging files. Can you show stderr? I suspect we're printing the exit
> message there.
>
> See: https://issues.apache.org/jira/browse/MESOS-5854
>
> On Fri, Jul 29, 2016 at 5:57 PM, Douglas Nelson 
> wrote:
>
>> It might be an issue with the mesos-init-wrapper? I'm using that to set
>> the flag via config files. I'll have to look through it and see exactly
>> what it's doing when it sets the attributes flag.
>>
>> On Fri, Jul 29, 2016 at 6:48 PM, Douglas Nelson 
>> wrote:
>>
>>> I'm pretty sure I set the flag right. Here is the agent's info:
>>>
>>> Log file created at: 2016/07/29 18:25:16
>>> Running on machine: lubuntu
>>> Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
>>> I0729 18:25:16.494326  4559 logging.cpp:194] INFO level logging started!
>>> I0729 18:25:16.496150  4559 containerizer.cpp:196] Using isolation:
>>> posix/cpu,posix/mem,filesystem/posix,network/cni
>>> I0729 18:25:16.498539  4559 linux_launcher.cpp:101] Using
>>> /sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
>>> I0729 18:25:16.499295  4559 main.cpp:434] Starting Mesos agent
>>> I0729 18:25:16.500417  4577 slave.cpp:198] Agent started on 1)@
>>> 127.0.1.1:5051
>>> I0729 18:25:16.500427  4577 slave.cpp:199] Flags at startup:
>>> --appc_simple_discovery_uri_prefix="http://;
>>> --appc_store_dir="/tmp/mesos/store/appc" --attributes="test:test"
>>> --authenticate_http_readonl...
>>> I0729 18:25:16.500751  4577 slave.cpp:519] Agent resources: cpus(*):1;
>>> mem(*):1000; disk(*):13901; ports(*):[31000-32000]
>>> I0729 18:25:16.500776  4577 slave.cpp:527] Agent attributes: [ test=test
>>> ]
>>> I0729 18:25:16.500779  4577 slave.cpp:532] Agent hostname: lubuntu
>>> I0729 18:25:16.502638  4578 state.cpp:57] Recovering state from
>>> '/var/lib/mesos/meta'
>>> I0729 18:25:16.502667  4578 state.cpp:697] No checkpointed resources
>>> found at '/var/lib/mesos/meta/resources/resources.info'
>>>
>>>
>>> On Fri, Jul 29, 2016 at 6:41 PM, Joseph Wu  wrote:
>>>
 Works fine for me.  Make sure the agent isn't just complaining about
 invalid flags.

 i.e. This is invalid:
 --attributes="something"

 This is valid:
 --attributes="something:foo"
 --attributes="something:foo; nothing:bar"

 And make sure your agent's work directory doesn't contain info from an
 agent started with different attributes (or no attributes).

 On Fri, Jul 29, 2016 at 5:31 PM, Douglas Nelson 
 wrote:

> When I set any attributes for the agent node it fails to run. No
> mesos-slave.ERROR log is created. I am using mesos 1.0.0 from the
> mesosphere package, but I also tried building it and had the same issue.
>
> As soon as I remove the --attributes flag the agent runs normally and
> registers itself with the master node. Is attributes deprecated? Is anyone
> else running into this?
>


>>>
>>
>

Re: Attributes cause agent to fail

[Benjamin Mahler]

Unfortunately we log termination messages to stderr rather than the logging
files. Can you show stderr? I suspect we're printing the exit message there.

See: https://issues.apache.org/jira/browse/MESOS-5854

On Fri, Jul 29, 2016 at 5:57 PM, Douglas Nelson  wrote:

> It might be an issue with the mesos-init-wrapper? I'm using that to set
> the flag via config files. I'll have to look through it and see exactly
> what it's doing when it sets the attributes flag.
>
> On Fri, Jul 29, 2016 at 6:48 PM, Douglas Nelson 
> wrote:
>
>> I'm pretty sure I set the flag right. Here is the agent's info:
>>
>> Log file created at: 2016/07/29 18:25:16
>> Running on machine: lubuntu
>> Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
>> I0729 18:25:16.494326  4559 logging.cpp:194] INFO level logging started!
>> I0729 18:25:16.496150  4559 containerizer.cpp:196] Using isolation:
>> posix/cpu,posix/mem,filesystem/posix,network/cni
>> I0729 18:25:16.498539  4559 linux_launcher.cpp:101] Using
>> /sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
>> I0729 18:25:16.499295  4559 main.cpp:434] Starting Mesos agent
>> I0729 18:25:16.500417  4577 slave.cpp:198] Agent started on 1)@
>> 127.0.1.1:5051
>> I0729 18:25:16.500427  4577 slave.cpp:199] Flags at startup:
>> --appc_simple_discovery_uri_prefix="http://;
>> --appc_store_dir="/tmp/mesos/store/appc" --attributes="test:test"
>> --authenticate_http_readonl...
>> I0729 18:25:16.500751  4577 slave.cpp:519] Agent resources: cpus(*):1;
>> mem(*):1000; disk(*):13901; ports(*):[31000-32000]
>> I0729 18:25:16.500776  4577 slave.cpp:527] Agent attributes: [ test=test ]
>> I0729 18:25:16.500779  4577 slave.cpp:532] Agent hostname: lubuntu
>> I0729 18:25:16.502638  4578 state.cpp:57] Recovering state from
>> '/var/lib/mesos/meta'
>> I0729 18:25:16.502667  4578 state.cpp:697] No checkpointed resources
>> found at '/var/lib/mesos/meta/resources/resources.info'
>>
>>
>> On Fri, Jul 29, 2016 at 6:41 PM, Joseph Wu  wrote:
>>
>>> Works fine for me.  Make sure the agent isn't just complaining about
>>> invalid flags.
>>>
>>> i.e. This is invalid:
>>> --attributes="something"
>>>
>>> This is valid:
>>> --attributes="something:foo"
>>> --attributes="something:foo; nothing:bar"
>>>
>>> And make sure your agent's work directory doesn't contain info from an
>>> agent started with different attributes (or no attributes).
>>>
>>> On Fri, Jul 29, 2016 at 5:31 PM, Douglas Nelson 
>>> wrote:
>>>
 When I set any attributes for the agent node it fails to run. No
 mesos-slave.ERROR log is created. I am using mesos 1.0.0 from the
 mesosphere package, but I also tried building it and had the same issue.

 As soon as I remove the --attributes flag the agent runs normally and
 registers itself with the master node. Is attributes deprecated? Is anyone
 else running into this?

>>>
>>>
>>
>

Re: Attributes cause agent to fail

[Douglas Nelson]

It might be an issue with the mesos-init-wrapper? I'm using that to set the
flag via config files. I'll have to look through it and see exactly what
it's doing when it sets the attributes flag.

On Fri, Jul 29, 2016 at 6:48 PM, Douglas Nelson  wrote:

> I'm pretty sure I set the flag right. Here is the agent's info:
>
> Log file created at: 2016/07/29 18:25:16
> Running on machine: lubuntu
> Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
> I0729 18:25:16.494326  4559 logging.cpp:194] INFO level logging started!
> I0729 18:25:16.496150  4559 containerizer.cpp:196] Using isolation:
> posix/cpu,posix/mem,filesystem/posix,network/cni
> I0729 18:25:16.498539  4559 linux_launcher.cpp:101] Using
> /sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
> I0729 18:25:16.499295  4559 main.cpp:434] Starting Mesos agent
> I0729 18:25:16.500417  4577 slave.cpp:198] Agent started on 1)@
> 127.0.1.1:5051
> I0729 18:25:16.500427  4577 slave.cpp:199] Flags at startup:
> --appc_simple_discovery_uri_prefix="http://;
> --appc_store_dir="/tmp/mesos/store/appc" --attributes="test:test"
> --authenticate_http_readonl...
> I0729 18:25:16.500751  4577 slave.cpp:519] Agent resources: cpus(*):1;
> mem(*):1000; disk(*):13901; ports(*):[31000-32000]
> I0729 18:25:16.500776  4577 slave.cpp:527] Agent attributes: [ test=test ]
> I0729 18:25:16.500779  4577 slave.cpp:532] Agent hostname: lubuntu
> I0729 18:25:16.502638  4578 state.cpp:57] Recovering state from
> '/var/lib/mesos/meta'
> I0729 18:25:16.502667  4578 state.cpp:697] No checkpointed resources found
> at '/var/lib/mesos/meta/resources/resources.info'
>
>
> On Fri, Jul 29, 2016 at 6:41 PM, Joseph Wu  wrote:
>
>> Works fine for me.  Make sure the agent isn't just complaining about
>> invalid flags.
>>
>> i.e. This is invalid:
>> --attributes="something"
>>
>> This is valid:
>> --attributes="something:foo"
>> --attributes="something:foo; nothing:bar"
>>
>> And make sure your agent's work directory doesn't contain info from an
>> agent started with different attributes (or no attributes).
>>
>> On Fri, Jul 29, 2016 at 5:31 PM, Douglas Nelson 
>> wrote:
>>
>>> When I set any attributes for the agent node it fails to run. No
>>> mesos-slave.ERROR log is created. I am using mesos 1.0.0 from the
>>> mesosphere package, but I also tried building it and had the same issue.
>>>
>>> As soon as I remove the --attributes flag the agent runs normally and
>>> registers itself with the master node. Is attributes deprecated? Is anyone
>>> else running into this?
>>>
>>
>>
>

Re: Attributes cause agent to fail

[Douglas Nelson]

I'm pretty sure I set the flag right. Here is the agent's info:

Log file created at: 2016/07/29 18:25:16
Running on machine: lubuntu
Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
I0729 18:25:16.494326  4559 logging.cpp:194] INFO level logging started!
I0729 18:25:16.496150  4559 containerizer.cpp:196] Using isolation:
posix/cpu,posix/mem,filesystem/posix,network/cni
I0729 18:25:16.498539  4559 linux_launcher.cpp:101] Using
/sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
I0729 18:25:16.499295  4559 main.cpp:434] Starting Mesos agent
I0729 18:25:16.500417  4577 slave.cpp:198] Agent started on 1)@
127.0.1.1:5051
I0729 18:25:16.500427  4577 slave.cpp:199] Flags at startup:
--appc_simple_discovery_uri_prefix="http://;
--appc_store_dir="/tmp/mesos/store/appc" --attributes="test:test"
--authenticate_http_readonl...
I0729 18:25:16.500751  4577 slave.cpp:519] Agent resources: cpus(*):1;
mem(*):1000; disk(*):13901; ports(*):[31000-32000]
I0729 18:25:16.500776  4577 slave.cpp:527] Agent attributes: [ test=test ]
I0729 18:25:16.500779  4577 slave.cpp:532] Agent hostname: lubuntu
I0729 18:25:16.502638  4578 state.cpp:57] Recovering state from
'/var/lib/mesos/meta'
I0729 18:25:16.502667  4578 state.cpp:697] No checkpointed resources found
at '/var/lib/mesos/meta/resources/resources.info'


On Fri, Jul 29, 2016 at 6:41 PM, Joseph Wu  wrote:

> Works fine for me.  Make sure the agent isn't just complaining about
> invalid flags.
>
> i.e. This is invalid:
> --attributes="something"
>
> This is valid:
> --attributes="something:foo"
> --attributes="something:foo; nothing:bar"
>
> And make sure your agent's work directory doesn't contain info from an
> agent started with different attributes (or no attributes).
>
> On Fri, Jul 29, 2016 at 5:31 PM, Douglas Nelson 
> wrote:
>
>> When I set any attributes for the agent node it fails to run. No
>> mesos-slave.ERROR log is created. I am using mesos 1.0.0 from the
>> mesosphere package, but I also tried building it and had the same issue.
>>
>> As soon as I remove the --attributes flag the agent runs normally and
>> registers itself with the master node. Is attributes deprecated? Is anyone
>> else running into this?
>>
>
>

Re: Attributes cause agent to fail

[Joseph Wu]

Works fine for me.  Make sure the agent isn't just complaining about
invalid flags.

i.e. This is invalid:
--attributes="something"

This is valid:
--attributes="something:foo"
--attributes="something:foo; nothing:bar"

And make sure your agent's work directory doesn't contain info from an
agent started with different attributes (or no attributes).

On Fri, Jul 29, 2016 at 5:31 PM, Douglas Nelson  wrote:

> When I set any attributes for the agent node it fails to run. No
> mesos-slave.ERROR log is created. I am using mesos 1.0.0 from the
> mesosphere package, but I also tried building it and had the same issue.
>
> As soon as I remove the --attributes flag the agent runs normally and
> registers itself with the master node. Is attributes deprecated? Is anyone
> else running into this?
>

Attributes cause agent to fail

[Douglas Nelson]

When I set any attributes for the agent node it fails to run. No
mesos-slave.ERROR log is created. I am using mesos 1.0.0 from the
mesosphere package, but I also tried building it and had the same issue.

As soon as I remove the --attributes flag the agent runs normally and
registers itself with the master node. Is attributes deprecated? Is anyone
else running into this?