Just for completeness and to provide an alternative, you can also probably leverage the dcos command line tool (https://github.com/dcos/dcos-cli) to get all the info you would need in a JSON format.
e.g., 1. set up ~/.dcos/config.toml for your cluster 2. DCOS_CONFIG=~/.dcos/config.toml dcos task --json --completed You could process that output with jq (https://stedolan.github.io/jq/) and do all of this in a short script. (Not that I have much luck using jq, I'm not very skilled at working with such arcane syntax, a la XPath.) - Erik On Wed, Aug 10, 2016 at 9:27 AM, June Taylor <[email protected]> wrote: > David, > > Thanks for the suggestions, this has the missing piece we needed! > > > Thanks, > June Taylor > System Administrator, Minnesota Population Center > University of Minnesota > > On Wed, Aug 10, 2016 at 10:54 AM, David Greenberg <[email protected]> > wrote: > >> The Cook framework has examples of how to do this. See here ( >> https://github.com/twosigma/Cook/blob/master/scheduler/src/ >> cook/mesos/api.clj#L322-L324) for constructing the stem of the URL, here >> (https://github.com/twosigma/Cook/blob/7a49fbb98b281e3b23779 >> cd88d1d2b73428a0447/scheduler/src/cook/mesos/api.clj#L281-L297) for >> finding the filesystem path, and here (https://github.com/twosigma/C >> ook/blob/master/scheduler/docs/scheduler-rest-api.asc#using_output_url) >> for getting data with the stem. >> >> Essentially, you need to scrape the Mesos master to get all the path info >> you need. LMK if you have questions! >> >> On Wed, Aug 10, 2016 at 9:34 AM June Taylor <[email protected]> wrote: >> >>> Tomek, >>> >>> I'm not sure I understand your suggestion. We know how to ask for a file >>> from an HTTP endpoint, but it is the construction of the correct URL which >>> is not currently clear. >>> >>> We are not sure how to determine the Run ID of the executor. >>> >>> >>> Thanks, >>> June Taylor >>> System Administrator, Minnesota Population Center >>> University of Minnesota >>> >>> On Wed, Aug 10, 2016 at 10:08 AM, Tomek Janiszewski <[email protected]> >>> wrote: >>> >>>> Hi >>>> >>>> If you need simplest method then python SimpleHTTPServer could help. >>>> Just launch it in background before command you want to run, assign it port >>>> and query sandbox with <agent ip>:<task port> that can be obtained from >>>> state endpoint. >>>> >>>> - >>>> Tomek >>>> >>>> śr., 10.08.2016 o 16:53 użytkownik June Taylor <[email protected]> napisał: >>>> >>>>> We are trying to retrieve the stdout and stderr files from an executor >>>>> programmatically. >>>>> >>>>> It appears that these are available via HTTP request, however, >>>>> constructing the correct URL is posing to be a challenge. >>>>> >>>>> Our scenario is: >>>>> >>>>> 1. Use mesos-execute to submit a job. A framework ID is available at >>>>> this point. >>>>> 2. Using the framework ID, one can inquire with mesos-state to >>>>> determine which slave ID is executing the task. >>>>> 3. Using the slave ID, one can inquire with mesos-state to find the >>>>> hostname for that slave ID >>>>> 4. HTTP can be used to ask the /browse/ endpoint for a file, however, >>>>> there is an Executor ID which we cannot programmatically determine, to >>>>> complete this URL. >>>>> >>>>> Please advise the simplest option for retrieving the sandbox files >>>>> given the scenario starts with mesos-execute commands. >>>>> >>>>> Thanks! >>>>> June Taylor >>>>> System Administrator, Minnesota Population Center >>>>> University of Minnesota >>>>> >>>> >>> >
