David, Thanks for the suggestions, this has the missing piece we needed!
Thanks, June Taylor System Administrator, Minnesota Population Center University of Minnesota On Wed, Aug 10, 2016 at 10:54 AM, David Greenberg <[email protected]> wrote: > The Cook framework has examples of how to do this. See here ( > https://github.com/twosigma/Cook/blob/master/scheduler/ > src/cook/mesos/api.clj#L322-L324) for constructing the stem of the URL, > here (https://github.com/twosigma/Cook/blob/7a49fbb98b281e3b23779cd88d1d2b > 73428a0447/scheduler/src/cook/mesos/api.clj#L281-L297) for finding the > filesystem path, and here (https://github.com/twosigma/ > Cook/blob/master/scheduler/docs/scheduler-rest-api.asc#using_output_url) > for getting data with the stem. > > Essentially, you need to scrape the Mesos master to get all the path info > you need. LMK if you have questions! > > On Wed, Aug 10, 2016 at 9:34 AM June Taylor <[email protected]> wrote: > >> Tomek, >> >> I'm not sure I understand your suggestion. We know how to ask for a file >> from an HTTP endpoint, but it is the construction of the correct URL which >> is not currently clear. >> >> We are not sure how to determine the Run ID of the executor. >> >> >> Thanks, >> June Taylor >> System Administrator, Minnesota Population Center >> University of Minnesota >> >> On Wed, Aug 10, 2016 at 10:08 AM, Tomek Janiszewski <[email protected]> >> wrote: >> >>> Hi >>> >>> If you need simplest method then python SimpleHTTPServer could help. >>> Just launch it in background before command you want to run, assign it port >>> and query sandbox with <agent ip>:<task port> that can be obtained from >>> state endpoint. >>> >>> - >>> Tomek >>> >>> śr., 10.08.2016 o 16:53 użytkownik June Taylor <[email protected]> napisał: >>> >>>> We are trying to retrieve the stdout and stderr files from an executor >>>> programmatically. >>>> >>>> It appears that these are available via HTTP request, however, >>>> constructing the correct URL is posing to be a challenge. >>>> >>>> Our scenario is: >>>> >>>> 1. Use mesos-execute to submit a job. A framework ID is available at >>>> this point. >>>> 2. Using the framework ID, one can inquire with mesos-state to >>>> determine which slave ID is executing the task. >>>> 3. Using the slave ID, one can inquire with mesos-state to find the >>>> hostname for that slave ID >>>> 4. HTTP can be used to ask the /browse/ endpoint for a file, however, >>>> there is an Executor ID which we cannot programmatically determine, to >>>> complete this URL. >>>> >>>> Please advise the simplest option for retrieving the sandbox files >>>> given the scenario starts with mesos-execute commands. >>>> >>>> Thanks! >>>> June Taylor >>>> System Administrator, Minnesota Population Center >>>> University of Minnesota >>>> >>> >>
