First off, you can only do this per executor - not per task.
To my understanding, the webui just picks up the executor directory from
the slave state.json, but the container id should also be available there.
Here is a snippet from a run of a echo 'foobar' > baz.txt job:
The slave state endpoint:
{
...
"completed_frameworks": [
{
...
"completed_executors": [
{
...
"*container*": "26491475-9515-464d-bcd0-c72790489a85",
"*directory*":
"/tmp/mesos/0/slaves/20150122-175457-16777343-5050-18106-S0/frameworks/20150122-182305-16777343-5050-18833-0000/executors/bar/runs/26491475-9515-464d-bcd0-c72790489a85",
"id": "bar",
"name": "Command Executor (Task: bar) (Command: sh -c 'echo
'foobar...')",
...
}
],
"executors": [],
...
}
],
...
"frameworks": [],
...
}
>From that point on, you can browse (path 'just' URL encoded):
$ curl http://
<host>:5050/files/browse.json?path=%2Ftmp%2Fmesos%2F0%2Fslaves%2F20150122-175457-16777343-5050-18106-S0%2Fframeworks%2F20150122-182305-16777343-5050-18833-0000%2Fexecutors%2Fbar%2Fruns%2F26491475-9515-464d-bcd0-c72790489a85
[
{
"gid": "wheel",
"mode": "-rw-r--r--",
"mtime": 1421979805,
"nlink": 1,
"path":
"/tmp/mesos/0/slaves/20150122-175457-16777343-5050-18106-S0/frameworks/20150122-182305-16777343-5050-18833-0000/executors/bar/runs/26491475-9515-464d-bcd0-c72790489a85/baz.txt",
"size": 7,
"uid": "nnielsen"
}
]
Or grab the file:
$ curl
http://localhost:5050/files/read.json?path=%2Ftmp%2Fmesos%2F0%2Fslaves%2F20150122-175457-16777343-5050-18106-S0%2Fframeworks%2F20150122-182305-16777343-5050-18833-0000%2Fexecutors%2Fbar%2Fruns%2F26491475-9515-464d-bcd0-c72790489a85%2Fbaz.txt\&offset=0
{
"data": "foobar\n",
"offset": 0
}
Hope this helps.
Niklas
On 21 January 2015 at 13:53, David Greenberg <[email protected]> wrote:
> It seems that if I take the URL that the "Download" button for stderr
> points to and curl it, I get the file. But, if I change the container_id to
> "latest" instead of the UUID, then I get a 404. Is there another way to
> resolve what the container_id is, since it seems critical to get files
> programmatically.
>
> On Wed, Jan 21, 2015 at 3:17 PM, Ian Downes <[email protected]> wrote:
>
>> No, the container id is generated by the slave when it launches the
>> executor for a task (see Framework::launchExecutor() in
>> src/slave/slave.cpp).
>>
>> However, the 'latest' symlink will point to the most recent container_id
>> directory so you can likely just use that unless your framework is re-using
>> executor_ids (which would mean a new container for each run).
>>
>> On Wed, Jan 21, 2015 at 11:52 AM, David Greenberg <[email protected]
>> > wrote:
>>
>>> Is it possible to know the container_id prior when you submit the
>>> TaskInfo? If not, how can you find it out?
>>>
>>> On Wed, Jan 21, 2015 at 1:17 PM, Ian Downes <[email protected]> wrote:
>>>
>>>> The final component is the container_id. Take a look in
>>>> src/slave/paths.hpp to see the directory layout.
>>>>
>>>> On Wed, Jan 21, 2015 at 8:50 AM, David Greenberg <
>>>> [email protected]> wrote:
>>>>
>>>>> So, I've looked into this more, and the UUID in "runs" doesn't appear
>>>>> appear to be the task-id, executor-id, or framework-id. do you have any
>>>>> idea what it could be?
>>>>>
>>>>> On Tue, Jan 13, 2015 at 5:21 PM, David Greenberg <
>>>>> [email protected]> wrote:
>>>>>
>>>>>> Thank you for your answers!
>>>>>>
>>>>>> On Tue, Jan 13, 2015 at 5:15 PM, Tim Chen <[email protected]> wrote:
>>>>>>
>>>>>>> You can get the slave_id, framework_id and executor_id of a task all
>>>>>>> from state.json.
>>>>>>>
>>>>>>> ie:
>>>>>>>
>>>>>>>
>>>>>>> - {
>>>>>>> - executor_id: "20141231-115728-16777343-5050-49193-S0",
>>>>>>> - framework_id: "20141231-115728-16777343-5050-49193-0000",
>>>>>>> - id: "1",
>>>>>>> - labels: [ ],
>>>>>>> - name: "Task 1",
>>>>>>> - resources:
>>>>>>> {
>>>>>>> - cpus: 6,
>>>>>>> - disk: 0,
>>>>>>> - mem: 13312
>>>>>>> },
>>>>>>> - slave_id: "20141231-115728-16777343-5050-49193-S0",
>>>>>>> - state: "TASK_KILLED",
>>>>>>> - statuses:
>>>>>>> [
>>>>>>> -
>>>>>>> {
>>>>>>> - state: "TASK_RUNNING",
>>>>>>> - timestamp: 1420056049.88177
>>>>>>> },
>>>>>>> -
>>>>>>> {
>>>>>>> - state: "TASK_KILLED",
>>>>>>> - timestamp: 1420056124.66483
>>>>>>> }
>>>>>>> ]
>>>>>>> },
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Jan 13, 2015 at 1:48 PM, David Greenberg <
>>>>>>> [email protected]> wrote:
>>>>>>>
>>>>>>>> I was trying to figure out how to programmatically access a task's
>>>>>>>> stdout & stderr, and I don't fully understand how the URL is
>>>>>>>> constructed.
>>>>>>>> It seems to be of the form http://
>>>>>>>> $slave_url:5050/read.json?$work_dir/work/slaves/$slave_id/frameworks/$framework_id/executors/$executor_id/runs/$something
>>>>>>>>
>>>>>>>> What is the $something? Is there an easier way, given just the
>>>>>>>> task_id, to find where the output is?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> David
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
