The Cook framework has examples of how to do this. See here ( https://github.com/twosigma/Cook/blob/master/scheduler/src/cook/mesos/api.clj#L322-L324) for constructing the stem of the URL, here ( https://github.com/twosigma/Cook/blob/7a49fbb98b281e3b23779cd88d1d2b73428a0447/scheduler/src/cook/mesos/api.clj#L281-L297) for finding the filesystem path, and here ( https://github.com/twosigma/Cook/blob/master/scheduler/docs/scheduler-rest-api.asc#using_output_url) for getting data with the stem.
Essentially, you need to scrape the Mesos master to get all the path info you need. LMK if you have questions! On Wed, Aug 10, 2016 at 9:34 AM June Taylor <[email protected]> wrote: > Tomek, > > I'm not sure I understand your suggestion. We know how to ask for a file > from an HTTP endpoint, but it is the construction of the correct URL which > is not currently clear. > > We are not sure how to determine the Run ID of the executor. > > > Thanks, > June Taylor > System Administrator, Minnesota Population Center > University of Minnesota > > On Wed, Aug 10, 2016 at 10:08 AM, Tomek Janiszewski <[email protected]> > wrote: > >> Hi >> >> If you need simplest method then python SimpleHTTPServer could help. Just >> launch it in background before command you want to run, assign it port and >> query sandbox with <agent ip>:<task port> that can be obtained from state >> endpoint. >> >> - >> Tomek >> >> śr., 10.08.2016 o 16:53 użytkownik June Taylor <[email protected]> napisał: >> >>> We are trying to retrieve the stdout and stderr files from an executor >>> programmatically. >>> >>> It appears that these are available via HTTP request, however, >>> constructing the correct URL is posing to be a challenge. >>> >>> Our scenario is: >>> >>> 1. Use mesos-execute to submit a job. A framework ID is available at >>> this point. >>> 2. Using the framework ID, one can inquire with mesos-state to determine >>> which slave ID is executing the task. >>> 3. Using the slave ID, one can inquire with mesos-state to find the >>> hostname for that slave ID >>> 4. HTTP can be used to ask the /browse/ endpoint for a file, however, >>> there is an Executor ID which we cannot programmatically determine, to >>> complete this URL. >>> >>> Please advise the simplest option for retrieving the sandbox files given >>> the scenario starts with mesos-execute commands. >>> >>> Thanks! >>> June Taylor >>> System Administrator, Minnesota Population Center >>> University of Minnesota >>> >> >
