> > all standard mesos inputs are available to the custom executor which is > driven from client >
This is the part i'm not convinced is a good idea due to security issues. The unix user of the executor (via setuid) is set at the framework level [1]. This would force you to use a completely unprivileged user for the whole cluster to avoid escalation issues. Perhaps further down the line we could create an admin API for adding/removing executors, however. [1] https://github.com/apache/mesos/blob/1e7be5ca417871828e21c38de07142ee9b2c772e/include/mesos/mesos.proto#L208-L212 On Sun, Nov 8, 2015 at 10:18 AM, <meghdoo...@yahoo.com> wrote: > I see. That would be excellent Bill if you are willing to take on this and > actively drive it. > > As you mention if all standard mesos inputs are available to the custom > executor which is driven from client that would be great. URI set for files > in sandbox, data blob (could be anything that an executor wants from > client), set of dynamic ports send as resources to executor from scheduler > as indicated by client (some convention defined in client request that is > understood by scheduler), env variables etc that I can think of. > > Thx > > Sent from my iPhone > > On Nov 7, 2015, at 7:39 PM, Bill Farner <wfar...@apache.org> wrote: > > Could you kindly elaborate with an example how the end user can pass in >> the executor of their choice in the API . I remember that other than a name >> passed on the API there was no scope to pass in the path of the custom >> executor or things like Fetcher URI sets that can be downloaded by mesos in >> the sandbox. > > > The user submitting jobs cannot do this, but the cluster administrator can > specify an arbitrary executor for the cluster (which is effectively done > today by setting the URI of the thermos executor). In other words, i can > use a non-thermos executor by using the scheduler thrift API so long as the > executor data field is compatible with the executor referenced by the > scheduler command line. > > At any rate, i think adding support for custom executor fields (mirroring > mesos protobuf fields) is valuable. I would like to start there, and at > that point adding support for multiple executors becomes trivial. > > I'm happy to take this work on myself, though it will probably be another > week or two before patches start landing. I already have a patch locally > that does a bunch of necessary prep work, however. > > Thanks for pushing on this feature! > > > On Wed, Nov 4, 2015 at 8:50 AM, <meghdoo...@yahoo.com> wrote: > >> Could you kindly elaborate with an example how the end user can pass in >> the executor of their choice in the API . I remember that other than a name >> passed on the API there was no scope to pass in the path of the custom >> executor or things like Fetcher URI sets that can be downloaded by mesos in >> the sandbox. >> I remember we discussed to make the change like marathon where one can >> pass in all details of the executor from client but you had opposed citing >> security. That should be captured in the ticket. Hence, the patch was made >> to load up selected executors from the scheduler side and depending on the >> executor name passed in the API the relevant executor configs can be used >> and filled in task. >> >> So kind of confused. Examples will help. >> >> Thx >> >> Sent from my iPhone >> >> On Nov 4, 2015, at 8:17 AM, Bill Farner <wfar...@apache.org> wrote: >> >> My change in stance partially came from the realization that the >> scheduler is not currently coupled to the executor implementation. As a >> result, an API consumer (bypassing the client) could already use any >> executor implementation they choose. This could be interpreted as custom >> executor support, and i would like to know if that satisfies the general >> use case. >> >> I am slightly uneasy about supporting user-selected executors on the >> scheduler simply because of the complexity that comes with it (the >> configuration complexity is one, but there will also likely be a need for >> ACLs, and even greater complexity in the client to support >> multiple/arbitrary configuration input formats). >> >> So, my question becomes - is this (end-user-selected executors) a real >> use case today, or a hypothetical one? If it a real use case, some >> elaboration would be useful to make sure we are building the right software >> for it. >> >> On Tue, Nov 3, 2015 at 9:30 PM, <meghdoo...@yahoo.com> wrote: >> >>> Bill what do you think? >>> >>> Thx >>> >>> Sent from my iPhone >>> >>> > On Nov 2, 2015, at 1:00 PM, Zameer Manji <zma...@apache.org> wrote: >>> > >>> > +wfarner >>> > >>> > I believe Bill was heavily involved in reviewing the proposed patch and >>> > design. Bill, care to comment on what you think here? >>> > >>> >> On Mon, Nov 2, 2015 at 12:55 PM, <meghdoo...@yahoo.com.invalid> >>> wrote: >>> >> >>> >> Do we have a decision on this? >>> >> >>> >> >>> https://issues.apache.org/jira/plugins/servlet/mobile#issue/AURORA-1376 >>> >> >>> >> It would help to know where we stand on this. >>> >> >>> >> Thx >>> >> >>> >> >>> >> Sent from my iPhone >>> >> >>> >> -- >>> >> Zameer Manji >>> >> >>> >> >>> >> >> >
