Joseph,
I thought I tried that. So I must still not following the directions. Here
is what I have?
mesos master running on OS X 10.10.5 mesos 0.26
I perform the following curl operation below.
server reads credentials file
I0929 10:48:42.062871 291536896 credentials.hpp:37] Loading credentials for
authentication from '/etc/mesos-master/attributes/credentials'
I0929 10:48:42.065512 291536896 master.cpp:467] Using default 'crammd5'
authenticator
The result of trying to reserve is: Could not authenticate 'mesos-mach5-beta'
======= the credentials file is =======
{
"credentials": [
{
"principal": "mesos-mach5-beta",
"secret": "password"
}
]
}
===============================
========================User curl post to reserve a slave not a framework
=============
SLAVE_ID="efb748eb-e1ce-423d-a795-7589c92b2a32-S1"
OPERATOR_PRINCIPAL="mach5"
CPUS="3"
MESOS_HOST="scaaa979.us.oracle.com<http://scaaa979.us.oracle.com>:5050"
curl -u "mesos-mach5-beta:password" -d slaveId="$SLAVE_ID" -d @- -X POST
http://$MESOS_HOST/master/reserve <<HERE
resources=[
{
"name": "cpus",
"type": "SCALAR",
"scalar": { "value": 8 },
"role": "mach5",
"reservation": {
"principal": "mesos-mach5-beta"
}
},
{
"name": "mem",
"type": "SCALAR",
"scalar": { "value": 4096 },
"role": "mach5",
"reservation": {
"principal": "mesos-mach5-beta"
}
}
]
=================================================================
On Sep 29, 2015, at 12:34 PM, Joseph Wu
<[email protected]<mailto:[email protected]>> wrote:
Rinaldo,
The principle is taken from authentication, rather than from the body of the
resources. In this case, you'll be using Basic Authentication:
https://en.wikipedia.org/wiki/Basic_access_authentication#Client_side
With curl, you'd add something like: -H "Authorization: Basic
bWVzb3MtbWFjaDUtYmV0YTpwYXNzd29yZA=="
That base64 blurb is the encoded version of "mesos-mach5-beta:password".
~Joseph
On Mon, Sep 28, 2015 at 8:25 PM, DiGiorgio, Mr. Rinaldo S.
<[email protected]<mailto:[email protected]>> wrote:
On Sep 28, 2015, at 8:03 PM, Joseph Wu
<[email protected]<mailto:[email protected]>> wrote:
Hi Rinaldo,
I'd like to point out a small error in your ACLs.
If you want to specify "ANY", you should set the "type" field. i.e. For the
RegisterFramework ACL:
"register_frameworks": [
{
"principals": { "values": "mesos-mach5-beta" },
"roles": { "type": 1 }
}
]
Thanks — can’t keep my eyes open any more. This is the response I get to the
following request.
Invalid RESERVE operation: Cannot reserve resources without a principal.
The example shows -u principal:password in curl which is an auentycation string
for the browser so I am totally confused on how to provide a principal. The
documentation for the framework reserve
curl -i -d slaveId="$SLAVE_ID" -d @- -X POST http://$MESOS_HOST/master/reserve
<<HERE
resources=[
{
"name": "cpus",
"type": "SCALAR",
"scalar": { "value": 8 },
"role": "mach5",
"reservation": {
"principal": "mach5"
}
},
{
"name": "mem",
"type": "SCALAR",
"scalar": { "value": 4096 },
"role": "mach5",
"reservation": {
"principal": "mach5"
}
}
]
<<HERE
The ANY "type" is part of an enumeration, defined here:
https://github.com/apache/mesos/blob/master/include/mesos/authorizer/authorizer.proto#L33-L45
Hope that helps,
~Joseph
On Mon, Sep 28, 2015 at 2:51 PM, DiGiorgio, Mr. Rinaldo S.
<[email protected]<mailto:[email protected]>> wrote:
On Sep 28, 2015, at 5:27 PM, Marco Massenzio
<[email protected]<mailto:[email protected]>> wrote:
Hi Rinaldo,
sorry about the trouble you're having in getting this to work!
If I got this one right, the original requirement was...
I have some tasks that need to run on different types of agents.
for that, I think you can use either (or both) of `roles` and `attributes` (see
the Configuration doc [0] for more info).
If you would like to run a 0.24 Mesos on your Mac for testing, you could use
the Mesosphere published packages[1] or, if Vagrant is more your thing, feel
free to "take inspiration" form [2].
Marco,
Thanks — We are running 0.23, 0.24 and the current branch as of this
morning in three mesos environments with linux and mac nodes and working on
porting Solaris. We have had various issues with building but are past most of
them. We are making progess on the Solaris build and there is an issue with
libsvn-1 as you mentioned with OL7.
Why do we need Dynamic Reservations?
We are also working with the mesos-plugin 0.8 and 0.9 and would like to change
some of the behaviors of the plugin. One of the changes we want to make and we
may move this out of the meson-plugin into workflow plugin in jenkins is to be
able to reserve all the resources we need before we start a series of tasks.
That is what we want to use dynamic reservations for. There may be issues with
the jenkins workflow architecture in that “slaves” have to be requested via
plugins. Mesos is new and I am sure it will provide a framework to innovate
on all the following currently supported scheduling options in LSF.
Fair share, preemptive, backfill and SLA scheduling
High throughput scheduling
Multicluster scheduling
Topology-, resource-, and energy-aware scheduling
I am trying to ask for a reservation and maybe I just don’t understand the
definitions. I seem to be unsure about what a principal is. Maybe that is the
root of my current issue. Unfortunately I am also a teacher so I notice
things like I still can’t find a definition of principal on all those web pages.
Thanks for all the links below but Docker is not a good technology for us
because it has the usual linuxism’s runs best and mostly on Linux. Vagrant has
the same issues so we will have to put more ports on our list. Docker don’t
have separation that is equal to the task so we need to match the resources of
the machine to the size of the task and not share in some circumstances. Our
apps tend to open lots of ports and use advanced features of the operating
system that may not be supported in Docker native, but may actually work in
Docker on a VM. Containers have different definitions of separation.
Rinaldo
Finally, to build on OSX, you'll need to install libsvn-1 as described in [3].
I'm afraid I don't know enough about Dynamic Reservation to really be able to
help here; but I suspect that, if you run without authentication enabled, it
will accept any principal (did you try that already? what error did you get?)
Feel free to drop me a line if you're still having trouble.
[0] http://mesos.apache.org/documentation/latest/configuration/
[1] http://mesosphere.com/downloads
[2] https://github.com/massenz/zk-mesos/tree/develop/vagrant
[3] http://mesos.apache.org/gettingstarted/ (see the OSX section; in particular:
`$ brew install autoconf automake libtool subversion maven`)
Marco Massenzio
Distributed Systems Engineer
http://codetrips.com<http://codetrips.com/>
On Mon, Sep 28, 2015 at 1:59 PM, DiGiorgio, Mr. Rinaldo S.
<[email protected]<mailto:[email protected]>> wrote:
On Sep 21, 2015, at 7:33 PM, Guangya Liu
<[email protected]<mailto:[email protected]>> wrote:
HI Rinaldo,
I think that you can use dynamic reservation feature to achieve this: You can
launch your tasks after reservation succeeds. Actually, all of the dynamic
reservation feature with endpoint has been finished except ACL part, so you can
use this feature now if you do not care ACL part.
Thanks,
Hi Guangya,
I have bene trying to get dynamic reservations to work. I downloaded the latest
from git and created a small environment on OS X 10.10. I am trying to use
reservations and I am not making much progress. I tried to get it to work
without authentication and was unable to. I used the ANY option and it still
required a principal. I am unable to configure the master to work without
authentication. Do you have some simple configs for starting a master with no
authentication required so that it can be used to set dynamic reservations.
The output below is for authentication. I tried to authenticate from a slave
and it failed with a coredump.
I start mesos like this:
mesos-master.sh —ip=nnn,nnn,nnn,nnn --work_dir=/var/lib/mesos --acls=$BASE/acls
--credentials=$BASE/credentials
bash-3.2# cat attributes/acls
{
"register_frameworks": [
{
"principals": { "type": "mesos-mach5-beta" },
"roles": { "values": "ANY" }
}
],
"run_tasks": [
{
"principals": { "values": "ANY" },
"users": { "values": "ANY" }
}
],
"shutdown_frameworks": [
{
"principals": { "values": "mesos-mach5-beta" },
"framework_principals": { "values": "ANY" }
}
]
}
bash-3.2# cat attributes/credentials
{
"credentials": [
{
"principal": "mesos-mach5-beta",
"secret": "password"
}
]
}
When I try the following I am told I am not authorized.
Guangya
On Tue, Sep 22, 2015 at 6:32 AM, DiGiorgio, Mr. Rinaldo S.
<[email protected]<mailto:[email protected]>> wrote:
Hi,
I have some tasks that need to run on different types of agents. I don’t
want the tasks to run unless I am going to have all the resources. Can someone
suggest how I could accomplish that with mesos. I read about reservations
here: http://mesos.apache.org/documentation/latest/reservation/
I could iterate over all the resources I need and if I get them proceed.
Is that the only way to do it?
Any idea when coming soon will be available?
/reserve (Coming Soon)
Suppose we want to reserve 8 CPUs and 4096 MB of RAM for the ads role on a
slave with id=<slave_id>. We send an HTTP POST request to the /reserve HTTP
endpoint like so:
Rinaldo
