Also I believe the CLI work that Haris / Kevin have been doing would make this easy to do via the Mesos CLI (it's not integrated into the project yet).
On Wed, Aug 10, 2016 at 9:57 AM, Erik Weathers <[email protected]> wrote: > Just for completeness and to provide an alternative, you can also probably > leverage the dcos command line tool (https://github.com/dcos/dcos-cli) to > get all the info you would need in a JSON format. > > e.g., > 1. set up ~/.dcos/config.toml for your cluster > 2. DCOS_CONFIG=~/.dcos/config.toml dcos task --json --completed > > You could process that output with jq (https://stedolan.github.io/jq/) > and do all of this in a short script. (Not that I have much luck using jq, > I'm not very skilled at working with such arcane syntax, a la XPath.) > > - Erik > > On Wed, Aug 10, 2016 at 9:27 AM, June Taylor <[email protected]> wrote: > >> David, >> >> Thanks for the suggestions, this has the missing piece we needed! >> >> >> Thanks, >> June Taylor >> System Administrator, Minnesota Population Center >> University of Minnesota >> >> On Wed, Aug 10, 2016 at 10:54 AM, David Greenberg <[email protected] >> > wrote: >> >>> The Cook framework has examples of how to do this. See here ( >>> https://github.com/twosigma/Cook/blob/master/scheduler/src/ >>> cook/mesos/api.clj#L322-L324) for constructing the stem of the URL, >>> here (https://github.com/twosigma/Cook/blob/7a49fbb98b281e3b23779 >>> cd88d1d2b73428a0447/scheduler/src/cook/mesos/api.clj#L281-L297) for >>> finding the filesystem path, and here (https://github.com/twosigma/C >>> ook/blob/master/scheduler/docs/scheduler-rest-api.asc#using_output_url) >>> for getting data with the stem. >>> >>> Essentially, you need to scrape the Mesos master to get all the path >>> info you need. LMK if you have questions! >>> >>> On Wed, Aug 10, 2016 at 9:34 AM June Taylor <[email protected]> wrote: >>> >>>> Tomek, >>>> >>>> I'm not sure I understand your suggestion. We know how to ask for a >>>> file from an HTTP endpoint, but it is the construction of the correct URL >>>> which is not currently clear. >>>> >>>> We are not sure how to determine the Run ID of the executor. >>>> >>>> >>>> Thanks, >>>> June Taylor >>>> System Administrator, Minnesota Population Center >>>> University of Minnesota >>>> >>>> On Wed, Aug 10, 2016 at 10:08 AM, Tomek Janiszewski <[email protected]> >>>> wrote: >>>> >>>>> Hi >>>>> >>>>> If you need simplest method then python SimpleHTTPServer could help. >>>>> Just launch it in background before command you want to run, assign it >>>>> port >>>>> and query sandbox with <agent ip>:<task port> that can be obtained from >>>>> state endpoint. >>>>> >>>>> - >>>>> Tomek >>>>> >>>>> śr., 10.08.2016 o 16:53 użytkownik June Taylor <[email protected]> napisał: >>>>> >>>>>> We are trying to retrieve the stdout and stderr files from an >>>>>> executor programmatically. >>>>>> >>>>>> It appears that these are available via HTTP request, however, >>>>>> constructing the correct URL is posing to be a challenge. >>>>>> >>>>>> Our scenario is: >>>>>> >>>>>> 1. Use mesos-execute to submit a job. A framework ID is available at >>>>>> this point. >>>>>> 2. Using the framework ID, one can inquire with mesos-state to >>>>>> determine which slave ID is executing the task. >>>>>> 3. Using the slave ID, one can inquire with mesos-state to find the >>>>>> hostname for that slave ID >>>>>> 4. HTTP can be used to ask the /browse/ endpoint for a file, however, >>>>>> there is an Executor ID which we cannot programmatically determine, to >>>>>> complete this URL. >>>>>> >>>>>> Please advise the simplest option for retrieving the sandbox files >>>>>> given the scenario starts with mesos-execute commands. >>>>>> >>>>>> Thanks! >>>>>> June Taylor >>>>>> System Administrator, Minnesota Population Center >>>>>> University of Minnesota >>>>>> >>>>> >>>> >> >
