Re: Escaped characters in the 'env' argument passed to mesos-execute

[June Taylor]

Chris,

Thanks for the info - we do suspect it's abount mesos-execute, yes.


Thanks,
June Taylor
System Administrator, Minnesota Population Center
University of Minnesota

On Thu, Jun 30, 2016 at 1:10 PM, Chris Baker  wrote:

> I tested with "/usr/bin/env && /usr/bin/sleep 30" to give myself a chance
> to look at stdout before the scheduler removed itself. I noticed that the
> problem is still in place; all environment variables have quotes and
> escaped slashes.
>
> This isn't a problem with launching a similar job via marathon:
> {
>   "id": "/chris/test",
>   "cmd": "/usr/bin/env && /usr/bin/sleep 30",
>   "cpus": 0.1,
>   "mem": 128,
>   "disk": 0,
>   "instances": 1,
>   "env": {
> "MY_FILEPATH": "/home/john/temp_output/test.csv"
>   }
> }
>
> stdout:
> Registered executor on 10.0.2.56
> Starting task chris_test.4fa0ac52-3eed-11e6-99b5-06094a33b9e2
> sh -c '/usr/bin/env && /usr/bin/sleep 30'
> Forked command at 7174
> ...
> MY_FILEPATH=/home/john/temp_output/test.csv
> ...
>
> So, I would assume it's something in mesos-exec. I haven't found the code
> that does it, so I can't figure out how to subvert it.
>
> On Thu, Jun 30, 2016 at 1:52 PM haosdent  wrote:
>
>> May you show the content of  /home/john/mesos/error_msg.py  I would like
>> to try it in my machine instead of executing `env` command.
>>
>> On Fri, Jul 1, 2016 at 1:40 AM, June Taylor  wrote:
>>
>>> Both of these suggestions are still resulting in this content being seen
>>> by the running application: \\/home\\/john\\/temp_output\\/test.csv"
>>>
>>> Thank you for your continued suggestions. Please try them out yourself
>>> and let me know if you can get a non-escaped path to print out successfully
>>> in your executing program, and let me know what you use to do that.
>>>
>>>
>>> Thanks,
>>> June Taylor
>>> System Administrator, Minnesota Population Center
>>> University of Minnesota
>>>
>>> On Thu, Jun 30, 2016 at 12:28 PM, Chris Baker 
>>> wrote:
>>>
 +1

 On Thu, Jun 30, 2016 at 1:15 PM haosdent  wrote:

> Actually you could write your json into a file and pass the file path
> into the flag. For example
>
> ```
> $ cat /tmp/a.json
> {"MY_FILEPATH": "/home/john/temp_output/test.csv"}
> ```
>
> And use `file:///tmp/a.json` as the value of env flag.
>
> On Fri, Jul 1, 2016 at 12:40 AM, Erik Weathers 
> wrote:
>
>> Heh, fair enough.  I know for the master/agent daemons you can put
>> the config values into files, which makes this pretty clean (though
>> that probably depends on the OS/init-launcher-daemon you are using).  But
>> yeah, when using the CLI you have to be pretty careful.
>>
>> - Erik
>>
>> On Thu, Jun 30, 2016 at 9:31 AM, Chris Baker 
>> wrote:
>>
>>> On a side note requiring people to put JSON on the command line
>>> is a sadistic thing to do.
>>>
>>> On Thu, Jun 30, 2016 at 12:28 PM Erik Weathers <
>>> eweath...@groupon.com> wrote:
>>>
 +1 I would wrap every string in quotes...  otherwise your shell
 doesn't know what you mean.  i.e.,

 How is the shell supposed to know that you want this all to be 1
 string value for the --command parameter?

 --command=/home/john/anaconda3/bin/python
 /home/john/mesos/error_msg.py read

 Similarly how is the shell to know that you *don't* want to end
 this command and start a new one when you have a bare semicolon?

 --resources=cpus:1;mem:128

 - Erik

 On Thu, Jun 30, 2016 at 9:08 AM, haosdent 
 wrote:

> I use bash as well. How about
> ```
> mesos-execute --master=cluster.example.com:5050 --name=test-program
> --command="/home/john/anaconda3/bin/python
> /home/john/mesos/error_msg.py read" --resources="cpus:1;mem:128"
> --env='{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'
> ```
>
> On Fri, Jul 1, 2016 at 12:04 AM, June Taylor  wrote:
>
>> hasodent,
>>
>> We've tried your suggestion but it seems like that's not working
>> out:
>>
>> mesos-execute --master=cluster.example.com:5050
>> --name=test-program --command=/home/john/anaconda3/bin/python
>> /home/john/mesos/error_msg.py read --resources=cpus:1;mem:128
>> --env='{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'
>>
>> john@cluster:~/mesos$ Failed to load flag 'env': Failed to load
>> value ''{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'': syntax 
>> error
>> at line 1 near: '{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'
>>
>>
>> We're 

Re: Escaped characters in the 'env' argument passed to mesos-execute

[Chris Baker]

I tested with "/usr/bin/env && /usr/bin/sleep 30" to give myself a chance
to look at stdout before the scheduler removed itself. I noticed that the
problem is still in place; all environment variables have quotes and
escaped slashes.

This isn't a problem with launching a similar job via marathon:
{
  "id": "/chris/test",
  "cmd": "/usr/bin/env && /usr/bin/sleep 30",
  "cpus": 0.1,
  "mem": 128,
  "disk": 0,
  "instances": 1,
  "env": {
"MY_FILEPATH": "/home/john/temp_output/test.csv"
  }
}

stdout:
Registered executor on 10.0.2.56
Starting task chris_test.4fa0ac52-3eed-11e6-99b5-06094a33b9e2
sh -c '/usr/bin/env && /usr/bin/sleep 30'
Forked command at 7174
...
MY_FILEPATH=/home/john/temp_output/test.csv
...

So, I would assume it's something in mesos-exec. I haven't found the code
that does it, so I can't figure out how to subvert it.

On Thu, Jun 30, 2016 at 1:52 PM haosdent  wrote:

> May you show the content of  /home/john/mesos/error_msg.py  I would like
> to try it in my machine instead of executing `env` command.
>
> On Fri, Jul 1, 2016 at 1:40 AM, June Taylor  wrote:
>
>> Both of these suggestions are still resulting in this content being seen
>> by the running application: \\/home\\/john\\/temp_output\\/test.csv"
>>
>> Thank you for your continued suggestions. Please try them out yourself
>> and let me know if you can get a non-escaped path to print out successfully
>> in your executing program, and let me know what you use to do that.
>>
>>
>> Thanks,
>> June Taylor
>> System Administrator, Minnesota Population Center
>> University of Minnesota
>>
>> On Thu, Jun 30, 2016 at 12:28 PM, Chris Baker 
>> wrote:
>>
>>> +1
>>>
>>> On Thu, Jun 30, 2016 at 1:15 PM haosdent  wrote:
>>>
 Actually you could write your json into a file and pass the file path
 into the flag. For example

 ```
 $ cat /tmp/a.json
 {"MY_FILEPATH": "/home/john/temp_output/test.csv"}
 ```

 And use `file:///tmp/a.json` as the value of env flag.

 On Fri, Jul 1, 2016 at 12:40 AM, Erik Weathers 
 wrote:

> Heh, fair enough.  I know for the master/agent daemons you can put the
> config values into files, which makes this pretty clean (though
> that probably depends on the OS/init-launcher-daemon you are using).  But
> yeah, when using the CLI you have to be pretty careful.
>
> - Erik
>
> On Thu, Jun 30, 2016 at 9:31 AM, Chris Baker 
> wrote:
>
>> On a side note requiring people to put JSON on the command line
>> is a sadistic thing to do.
>>
>> On Thu, Jun 30, 2016 at 12:28 PM Erik Weathers 
>> wrote:
>>
>>> +1 I would wrap every string in quotes...  otherwise your shell
>>> doesn't know what you mean.  i.e.,
>>>
>>> How is the shell supposed to know that you want this all to be 1
>>> string value for the --command parameter?
>>>
>>> --command=/home/john/anaconda3/bin/python
>>> /home/john/mesos/error_msg.py read
>>>
>>> Similarly how is the shell to know that you *don't* want to end this
>>> command and start a new one when you have a bare semicolon?
>>>
>>> --resources=cpus:1;mem:128
>>>
>>> - Erik
>>>
>>> On Thu, Jun 30, 2016 at 9:08 AM, haosdent 
>>> wrote:
>>>
 I use bash as well. How about
 ```
 mesos-execute --master=cluster.example.com:5050 --name=test-program
 --command="/home/john/anaconda3/bin/python
 /home/john/mesos/error_msg.py read" --resources="cpus:1;mem:128"
 --env='{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'
 ```

 On Fri, Jul 1, 2016 at 12:04 AM, June Taylor  wrote:

> hasodent,
>
> We've tried your suggestion but it seems like that's not working
> out:
>
> mesos-execute --master=cluster.example.com:5050
> --name=test-program --command=/home/john/anaconda3/bin/python
> /home/john/mesos/error_msg.py read --resources=cpus:1;mem:128
> --env='{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'
>
> john@cluster:~/mesos$ Failed to load flag 'env': Failed to load
> value ''{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'': syntax 
> error
> at line 1 near: '{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'
>
>
> We're using the Bash shell. Is anyone else passing file-paths
> within their environment variable argument?
>
>
> Thanks,
> June Taylor
> System Administrator, Minnesota Population Center
> University of Minnesota
>
> On Wed, Jun 29, 2016 at 8:55 PM, haosdent 
> wrote:
>
>> Yes, need add quotes for 

Re: Escaped characters in the 'env' argument passed to mesos-execute

[haosdent]

May you show the content of  /home/john/mesos/error_msg.py  I would like to
try it in my machine instead of executing `env` command.

On Fri, Jul 1, 2016 at 1:40 AM, June Taylor  wrote:

> Both of these suggestions are still resulting in this content being seen
> by the running application: \\/home\\/john\\/temp_output\\/test.csv"
>
> Thank you for your continued suggestions. Please try them out yourself and
> let me know if you can get a non-escaped path to print out successfully in
> your executing program, and let me know what you use to do that.
>
>
> Thanks,
> June Taylor
> System Administrator, Minnesota Population Center
> University of Minnesota
>
> On Thu, Jun 30, 2016 at 12:28 PM, Chris Baker 
> wrote:
>
>> +1
>>
>> On Thu, Jun 30, 2016 at 1:15 PM haosdent  wrote:
>>
>>> Actually you could write your json into a file and pass the file path
>>> into the flag. For example
>>>
>>> ```
>>> $ cat /tmp/a.json
>>> {"MY_FILEPATH": "/home/john/temp_output/test.csv"}
>>> ```
>>>
>>> And use `file:///tmp/a.json` as the value of env flag.
>>>
>>> On Fri, Jul 1, 2016 at 12:40 AM, Erik Weathers 
>>> wrote:
>>>
 Heh, fair enough.  I know for the master/agent daemons you can put the
 config values into files, which makes this pretty clean (though
 that probably depends on the OS/init-launcher-daemon you are using).  But
 yeah, when using the CLI you have to be pretty careful.

 - Erik

 On Thu, Jun 30, 2016 at 9:31 AM, Chris Baker 
 wrote:

> On a side note requiring people to put JSON on the command line is
> a sadistic thing to do.
>
> On Thu, Jun 30, 2016 at 12:28 PM Erik Weathers 
> wrote:
>
>> +1 I would wrap every string in quotes...  otherwise your shell
>> doesn't know what you mean.  i.e.,
>>
>> How is the shell supposed to know that you want this all to be 1
>> string value for the --command parameter?
>>
>> --command=/home/john/anaconda3/bin/python
>> /home/john/mesos/error_msg.py read
>>
>> Similarly how is the shell to know that you *don't* want to end this
>> command and start a new one when you have a bare semicolon?
>>
>> --resources=cpus:1;mem:128
>>
>> - Erik
>>
>> On Thu, Jun 30, 2016 at 9:08 AM, haosdent  wrote:
>>
>>> I use bash as well. How about
>>> ```
>>> mesos-execute --master=cluster.example.com:5050 --name=test-program
>>> --command="/home/john/anaconda3/bin/python
>>> /home/john/mesos/error_msg.py read" --resources="cpus:1;mem:128"
>>> --env='{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'
>>> ```
>>>
>>> On Fri, Jul 1, 2016 at 12:04 AM, June Taylor  wrote:
>>>
 hasodent,

 We've tried your suggestion but it seems like that's not working
 out:

 mesos-execute --master=cluster.example.com:5050
 --name=test-program --command=/home/john/anaconda3/bin/python
 /home/john/mesos/error_msg.py read --resources=cpus:1;mem:128
 --env='{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'

 john@cluster:~/mesos$ Failed to load flag 'env': Failed to load
 value ''{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'': syntax 
 error
 at line 1 near: '{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'


 We're using the Bash shell. Is anyone else passing file-paths
 within their environment variable argument?


 Thanks,
 June Taylor
 System Administrator, Minnesota Population Center
 University of Minnesota

 On Wed, Jun 29, 2016 at 8:55 PM, haosdent 
 wrote:

> Yes, need add quotes for --env='{"key": "value"}' flag.
>
> On Thu, Jun 30, 2016 at 4:39 AM, Chris Baker <
> ch...@galacticfog.com> wrote:
>
>> It's your shell:
>>
>> $ echo {"MY_FILEPATH": "/home/john/temp_output/test.csv"}
>> {MY_FILEPATH: /home/john/temp_output/test.csv}
>>
>> On Wed, Jun 29, 2016 at 4:05 PM June Taylor  wrote:
>>
>>> hasodent,
>>>
>>> I admit I don't see much of a difference in your version. Are
>>> you saying that Mesos doesn't escape the values, and that our 
>>> program may
>>> be doing that on the executor end?
>>>
>>>
>>> Thanks,
>>> June Taylor
>>> System Administrator, Minnesota Population Center
>>> University of Minnesota
>>>
>>> On Wed, Jun 29, 2016 at 2:02 PM, haosdent 
>>> wrote:
>>>
 Hi, @Taylor I use

 ```
 ./src/mesos-execute 

Re: Escaped characters in the 'env' argument passed to mesos-execute

[June Taylor]

Both of these suggestions are still resulting in this content being seen by
the running application: \\/home\\/john\\/temp_output\\/test.csv"

Thank you for your continued suggestions. Please try them out yourself and
let me know if you can get a non-escaped path to print out successfully in
your executing program, and let me know what you use to do that.


Thanks,
June Taylor
System Administrator, Minnesota Population Center
University of Minnesota

On Thu, Jun 30, 2016 at 12:28 PM, Chris Baker  wrote:

> +1
>
> On Thu, Jun 30, 2016 at 1:15 PM haosdent  wrote:
>
>> Actually you could write your json into a file and pass the file path
>> into the flag. For example
>>
>> ```
>> $ cat /tmp/a.json
>> {"MY_FILEPATH": "/home/john/temp_output/test.csv"}
>> ```
>>
>> And use `file:///tmp/a.json` as the value of env flag.
>>
>> On Fri, Jul 1, 2016 at 12:40 AM, Erik Weathers 
>> wrote:
>>
>>> Heh, fair enough.  I know for the master/agent daemons you can put the
>>> config values into files, which makes this pretty clean (though
>>> that probably depends on the OS/init-launcher-daemon you are using).  But
>>> yeah, when using the CLI you have to be pretty careful.
>>>
>>> - Erik
>>>
>>> On Thu, Jun 30, 2016 at 9:31 AM, Chris Baker 
>>> wrote:
>>>
 On a side note requiring people to put JSON on the command line is
 a sadistic thing to do.

 On Thu, Jun 30, 2016 at 12:28 PM Erik Weathers 
 wrote:

> +1 I would wrap every string in quotes...  otherwise your shell
> doesn't know what you mean.  i.e.,
>
> How is the shell supposed to know that you want this all to be 1
> string value for the --command parameter?
>
> --command=/home/john/anaconda3/bin/python
> /home/john/mesos/error_msg.py read
>
> Similarly how is the shell to know that you *don't* want to end this
> command and start a new one when you have a bare semicolon?
>
> --resources=cpus:1;mem:128
>
> - Erik
>
> On Thu, Jun 30, 2016 at 9:08 AM, haosdent  wrote:
>
>> I use bash as well. How about
>> ```
>> mesos-execute --master=cluster.example.com:5050 --name=test-program
>> --command="/home/john/anaconda3/bin/python
>> /home/john/mesos/error_msg.py read" --resources="cpus:1;mem:128"
>> --env='{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'
>> ```
>>
>> On Fri, Jul 1, 2016 at 12:04 AM, June Taylor  wrote:
>>
>>> hasodent,
>>>
>>> We've tried your suggestion but it seems like that's not working out:
>>>
>>> mesos-execute --master=cluster.example.com:5050 --name=test-program
>>> --command=/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
>>> read --resources=cpus:1;mem:128 --env='{"MY_FILEPATH":
>>> "/home/john/temp_output/test.csv"}'
>>>
>>> john@cluster:~/mesos$ Failed to load flag 'env': Failed to load
>>> value ''{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'': syntax 
>>> error
>>> at line 1 near: '{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'
>>>
>>>
>>> We're using the Bash shell. Is anyone else passing file-paths within
>>> their environment variable argument?
>>>
>>>
>>> Thanks,
>>> June Taylor
>>> System Administrator, Minnesota Population Center
>>> University of Minnesota
>>>
>>> On Wed, Jun 29, 2016 at 8:55 PM, haosdent 
>>> wrote:
>>>
 Yes, need add quotes for --env='{"key": "value"}' flag.

 On Thu, Jun 30, 2016 at 4:39 AM, Chris Baker  wrote:

> It's your shell:
>
> $ echo {"MY_FILEPATH": "/home/john/temp_output/test.csv"}
> {MY_FILEPATH: /home/john/temp_output/test.csv}
>
> On Wed, Jun 29, 2016 at 4:05 PM June Taylor  wrote:
>
>> hasodent,
>>
>> I admit I don't see much of a difference in your version. Are you
>> saying that Mesos doesn't escape the values, and that our program 
>> may be
>> doing that on the executor end?
>>
>>
>> Thanks,
>> June Taylor
>> System Administrator, Minnesota Population Center
>> University of Minnesota
>>
>> On Wed, Jun 29, 2016 at 2:02 PM, haosdent 
>> wrote:
>>
>>> Hi, @Taylor I use
>>>
>>> ```
>>> ./src/mesos-execute --env='{"MY_FILEPATH":
>>> "/home/john/temp_output/test.csv"}' --master="localhost:5050" 
>>> --command=env
>>> --name="test"
>>> ```
>>>
>>> to test in my env. The output looks correct in my side
>>>
>>> ```
>>> ...
>>> MY_FILEPATH=/home/john/temp_output/test.csv
>>>

Re: Escaped characters in the 'env' argument passed to mesos-execute

[Chris Baker]

+1

On Thu, Jun 30, 2016 at 1:15 PM haosdent  wrote:

> Actually you could write your json into a file and pass the file path into
> the flag. For example
>
> ```
> $ cat /tmp/a.json
> {"MY_FILEPATH": "/home/john/temp_output/test.csv"}
> ```
>
> And use `file:///tmp/a.json` as the value of env flag.
>
> On Fri, Jul 1, 2016 at 12:40 AM, Erik Weathers 
> wrote:
>
>> Heh, fair enough.  I know for the master/agent daemons you can put the
>> config values into files, which makes this pretty clean (though
>> that probably depends on the OS/init-launcher-daemon you are using).  But
>> yeah, when using the CLI you have to be pretty careful.
>>
>> - Erik
>>
>> On Thu, Jun 30, 2016 at 9:31 AM, Chris Baker 
>> wrote:
>>
>>> On a side note requiring people to put JSON on the command line is a
>>> sadistic thing to do.
>>>
>>> On Thu, Jun 30, 2016 at 12:28 PM Erik Weathers 
>>> wrote:
>>>
 +1 I would wrap every string in quotes...  otherwise your shell doesn't
 know what you mean.  i.e.,

 How is the shell supposed to know that you want this all to be 1 string
 value for the --command parameter?

 --command=/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
 read

 Similarly how is the shell to know that you *don't* want to end this
 command and start a new one when you have a bare semicolon?

 --resources=cpus:1;mem:128

 - Erik

 On Thu, Jun 30, 2016 at 9:08 AM, haosdent  wrote:

> I use bash as well. How about
> ```
> mesos-execute --master=cluster.example.com:5050 --name=test-program
> --command="/home/john/anaconda3/bin/python
> /home/john/mesos/error_msg.py read" --resources="cpus:1;mem:128"
> --env='{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'
> ```
>
> On Fri, Jul 1, 2016 at 12:04 AM, June Taylor  wrote:
>
>> hasodent,
>>
>> We've tried your suggestion but it seems like that's not working out:
>>
>> mesos-execute --master=cluster.example.com:5050 --name=test-program
>> --command=/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
>> read --resources=cpus:1;mem:128 --env='{"MY_FILEPATH":
>> "/home/john/temp_output/test.csv"}'
>>
>> john@cluster:~/mesos$ Failed to load flag 'env': Failed to load
>> value ''{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'': syntax 
>> error
>> at line 1 near: '{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'
>>
>>
>> We're using the Bash shell. Is anyone else passing file-paths within
>> their environment variable argument?
>>
>>
>> Thanks,
>> June Taylor
>> System Administrator, Minnesota Population Center
>> University of Minnesota
>>
>> On Wed, Jun 29, 2016 at 8:55 PM, haosdent  wrote:
>>
>>> Yes, need add quotes for --env='{"key": "value"}' flag.
>>>
>>> On Thu, Jun 30, 2016 at 4:39 AM, Chris Baker 
>>> wrote:
>>>
 It's your shell:

 $ echo {"MY_FILEPATH": "/home/john/temp_output/test.csv"}
 {MY_FILEPATH: /home/john/temp_output/test.csv}

 On Wed, Jun 29, 2016 at 4:05 PM June Taylor  wrote:

> hasodent,
>
> I admit I don't see much of a difference in your version. Are you
> saying that Mesos doesn't escape the values, and that our program may 
> be
> doing that on the executor end?
>
>
> Thanks,
> June Taylor
> System Administrator, Minnesota Population Center
> University of Minnesota
>
> On Wed, Jun 29, 2016 at 2:02 PM, haosdent 
> wrote:
>
>> Hi, @Taylor I use
>>
>> ```
>> ./src/mesos-execute --env='{"MY_FILEPATH":
>> "/home/john/temp_output/test.csv"}' --master="localhost:5050" 
>> --command=env
>> --name="test"
>> ```
>>
>> to test in my env. The output looks correct in my side
>>
>> ```
>> ...
>> MY_FILEPATH=/home/john/temp_output/test.csv
>>
>> MESOS_SANDBOX=/tmp/mesos/slaves/fe6123b9-8757-4015-be4b-fda901247fce-S0/frameworks/fe6123b9-8757-4015-be4b-fda901247fce-/executors/test/runs/fe818890-2a22-4b2b-aab2-816ed0b631d5
>> ...
>> ```
>>
>> Not sure if your problem caused by your program
>>
>> On Thu, Jun 30, 2016 at 1:58 AM, June Taylor 
>> wrote:
>>
>>> Greetings,
>>>
>>> We are using mesos-execute and passing an argument that contains
>>> a Unix filepath, as follows:
>>>
>>> *mesos-execute --master=cluster.example.com:5050
>>>  

Re: Escaped characters in the 'env' argument passed to mesos-execute

[haosdent]

Actually you could write your json into a file and pass the file path into
the flag. For example

```
$ cat /tmp/a.json
{"MY_FILEPATH": "/home/john/temp_output/test.csv"}
```

And use `file:///tmp/a.json` as the value of env flag.

On Fri, Jul 1, 2016 at 12:40 AM, Erik Weathers 
wrote:

> Heh, fair enough.  I know for the master/agent daemons you can put the
> config values into files, which makes this pretty clean (though
> that probably depends on the OS/init-launcher-daemon you are using).  But
> yeah, when using the CLI you have to be pretty careful.
>
> - Erik
>
> On Thu, Jun 30, 2016 at 9:31 AM, Chris Baker 
> wrote:
>
>> On a side note requiring people to put JSON on the command line is a
>> sadistic thing to do.
>>
>> On Thu, Jun 30, 2016 at 12:28 PM Erik Weathers 
>> wrote:
>>
>>> +1 I would wrap every string in quotes...  otherwise your shell doesn't
>>> know what you mean.  i.e.,
>>>
>>> How is the shell supposed to know that you want this all to be 1 string
>>> value for the --command parameter?
>>>
>>> --command=/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
>>> read
>>>
>>> Similarly how is the shell to know that you *don't* want to end this
>>> command and start a new one when you have a bare semicolon?
>>>
>>> --resources=cpus:1;mem:128
>>>
>>> - Erik
>>>
>>> On Thu, Jun 30, 2016 at 9:08 AM, haosdent  wrote:
>>>
 I use bash as well. How about
 ```
 mesos-execute --master=cluster.example.com:5050 --name=test-program
 --command="/home/john/anaconda3/bin/python
 /home/john/mesos/error_msg.py read" --resources="cpus:1;mem:128"
 --env='{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'
 ```

 On Fri, Jul 1, 2016 at 12:04 AM, June Taylor  wrote:

> hasodent,
>
> We've tried your suggestion but it seems like that's not working out:
>
> mesos-execute --master=cluster.example.com:5050 --name=test-program
> --command=/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
> read --resources=cpus:1;mem:128 --env='{"MY_FILEPATH":
> "/home/john/temp_output/test.csv"}'
>
> john@cluster:~/mesos$ Failed to load flag 'env': Failed to load value
> ''{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'': syntax error at
> line 1 near: '{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'
>
>
> We're using the Bash shell. Is anyone else passing file-paths within
> their environment variable argument?
>
>
> Thanks,
> June Taylor
> System Administrator, Minnesota Population Center
> University of Minnesota
>
> On Wed, Jun 29, 2016 at 8:55 PM, haosdent  wrote:
>
>> Yes, need add quotes for --env='{"key": "value"}' flag.
>>
>> On Thu, Jun 30, 2016 at 4:39 AM, Chris Baker 
>> wrote:
>>
>>> It's your shell:
>>>
>>> $ echo {"MY_FILEPATH": "/home/john/temp_output/test.csv"}
>>> {MY_FILEPATH: /home/john/temp_output/test.csv}
>>>
>>> On Wed, Jun 29, 2016 at 4:05 PM June Taylor  wrote:
>>>
 hasodent,

 I admit I don't see much of a difference in your version. Are you
 saying that Mesos doesn't escape the values, and that our program may 
 be
 doing that on the executor end?


 Thanks,
 June Taylor
 System Administrator, Minnesota Population Center
 University of Minnesota

 On Wed, Jun 29, 2016 at 2:02 PM, haosdent 
 wrote:

> Hi, @Taylor I use
>
> ```
> ./src/mesos-execute --env='{"MY_FILEPATH":
> "/home/john/temp_output/test.csv"}' --master="localhost:5050" 
> --command=env
> --name="test"
> ```
>
> to test in my env. The output looks correct in my side
>
> ```
> ...
> MY_FILEPATH=/home/john/temp_output/test.csv
>
> MESOS_SANDBOX=/tmp/mesos/slaves/fe6123b9-8757-4015-be4b-fda901247fce-S0/frameworks/fe6123b9-8757-4015-be4b-fda901247fce-/executors/test/runs/fe818890-2a22-4b2b-aab2-816ed0b631d5
> ...
> ```
>
> Not sure if your problem caused by your program
>
> On Thu, Jun 30, 2016 at 1:58 AM, June Taylor  wrote:
>
>> Greetings,
>>
>> We are using mesos-execute and passing an argument that contains
>> a Unix filepath, as follows:
>>
>> *mesos-execute --master=cluster.example.com:5050
>>  --name=test-program
>> --command=/home/john/anaconda3/bin/python 
>> /home/john/mesos/error_msg.py
>> read --resources=cpus:1;mem:128 --env={"MY_FILEPATH":
>> "/home/john/temp_output/test.csv"}*
>>

Re: Escaped characters in the 'env' argument passed to mesos-execute

[Erik Weathers]

Heh, fair enough.  I know for the master/agent daemons you can put the
config values into files, which makes this pretty clean (though
that probably depends on the OS/init-launcher-daemon you are using).  But
yeah, when using the CLI you have to be pretty careful.

- Erik

On Thu, Jun 30, 2016 at 9:31 AM, Chris Baker  wrote:

> On a side note requiring people to put JSON on the command line is a
> sadistic thing to do.
>
> On Thu, Jun 30, 2016 at 12:28 PM Erik Weathers 
> wrote:
>
>> +1 I would wrap every string in quotes...  otherwise your shell doesn't
>> know what you mean.  i.e.,
>>
>> How is the shell supposed to know that you want this all to be 1 string
>> value for the --command parameter?
>>
>> --command=/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
>> read
>>
>> Similarly how is the shell to know that you *don't* want to end this
>> command and start a new one when you have a bare semicolon?
>>
>> --resources=cpus:1;mem:128
>>
>> - Erik
>>
>> On Thu, Jun 30, 2016 at 9:08 AM, haosdent  wrote:
>>
>>> I use bash as well. How about
>>> ```
>>> mesos-execute --master=cluster.example.com:5050 --name=test-program
>>> --command="/home/john/anaconda3/bin/python
>>> /home/john/mesos/error_msg.py read" --resources="cpus:1;mem:128"
>>> --env='{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'
>>> ```
>>>
>>> On Fri, Jul 1, 2016 at 12:04 AM, June Taylor  wrote:
>>>
 hasodent,

 We've tried your suggestion but it seems like that's not working out:

 mesos-execute --master=cluster.example.com:5050 --name=test-program
 --command=/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
 read --resources=cpus:1;mem:128 --env='{"MY_FILEPATH":
 "/home/john/temp_output/test.csv"}'

 john@cluster:~/mesos$ Failed to load flag 'env': Failed to load value
 ''{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'': syntax error at
 line 1 near: '{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'


 We're using the Bash shell. Is anyone else passing file-paths within
 their environment variable argument?


 Thanks,
 June Taylor
 System Administrator, Minnesota Population Center
 University of Minnesota

 On Wed, Jun 29, 2016 at 8:55 PM, haosdent  wrote:

> Yes, need add quotes for --env='{"key": "value"}' flag.
>
> On Thu, Jun 30, 2016 at 4:39 AM, Chris Baker 
> wrote:
>
>> It's your shell:
>>
>> $ echo {"MY_FILEPATH": "/home/john/temp_output/test.csv"}
>> {MY_FILEPATH: /home/john/temp_output/test.csv}
>>
>> On Wed, Jun 29, 2016 at 4:05 PM June Taylor  wrote:
>>
>>> hasodent,
>>>
>>> I admit I don't see much of a difference in your version. Are you
>>> saying that Mesos doesn't escape the values, and that our program may be
>>> doing that on the executor end?
>>>
>>>
>>> Thanks,
>>> June Taylor
>>> System Administrator, Minnesota Population Center
>>> University of Minnesota
>>>
>>> On Wed, Jun 29, 2016 at 2:02 PM, haosdent 
>>> wrote:
>>>
 Hi, @Taylor I use

 ```
 ./src/mesos-execute --env='{"MY_FILEPATH":
 "/home/john/temp_output/test.csv"}' --master="localhost:5050" 
 --command=env
 --name="test"
 ```

 to test in my env. The output looks correct in my side

 ```
 ...
 MY_FILEPATH=/home/john/temp_output/test.csv

 MESOS_SANDBOX=/tmp/mesos/slaves/fe6123b9-8757-4015-be4b-fda901247fce-S0/frameworks/fe6123b9-8757-4015-be4b-fda901247fce-/executors/test/runs/fe818890-2a22-4b2b-aab2-816ed0b631d5
 ...
 ```

 Not sure if your problem caused by your program

 On Thu, Jun 30, 2016 at 1:58 AM, June Taylor  wrote:

> Greetings,
>
> We are using mesos-execute and passing an argument that contains a
> Unix filepath, as follows:
>
> *mesos-execute --master=cluster.example.com:5050
>  --name=test-program
> --command=/home/john/anaconda3/bin/python 
> /home/john/mesos/error_msg.py
> read --resources=cpus:1;mem:128 --env={"MY_FILEPATH":
> "/home/john/temp_output/test.csv"}*
>
> The slashes in the file path are then apparently escaped by
> mesos-execute and therefore fail to be useful once picked up by the
> executor:
>
> *FileNotFoundError: [Errno 2] No such file or directory:
> '"\\/home\\/john\\/temp_output\\/test.csv"'*
>
> *I0629 10:13:13.814870 14025 exec.cpp:390] Executor asked to
> shutdown*
>
> Are other people experiencing this? What is 

Re: Escaped characters in the 'env' argument passed to mesos-execute

[Chris Baker]

On a side note requiring people to put JSON on the command line is a
sadistic thing to do.

On Thu, Jun 30, 2016 at 12:28 PM Erik Weathers 
wrote:

> +1 I would wrap every string in quotes...  otherwise your shell doesn't
> know what you mean.  i.e.,
>
> How is the shell supposed to know that you want this all to be 1 string
> value for the --command parameter?
>
> --command=/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
> read
>
> Similarly how is the shell to know that you *don't* want to end this
> command and start a new one when you have a bare semicolon?
>
> --resources=cpus:1;mem:128
>
> - Erik
>
> On Thu, Jun 30, 2016 at 9:08 AM, haosdent  wrote:
>
>> I use bash as well. How about
>> ```
>> mesos-execute --master=cluster.example.com:5050 --name=test-program
>> --command="/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
>> read" --resources="cpus:1;mem:128" --env='{"MY_FILEPATH":
>> "/home/john/temp_output/test.csv"}'
>> ```
>>
>> On Fri, Jul 1, 2016 at 12:04 AM, June Taylor  wrote:
>>
>>> hasodent,
>>>
>>> We've tried your suggestion but it seems like that's not working out:
>>>
>>> mesos-execute --master=cluster.example.com:5050 --name=test-program
>>> --command=/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
>>> read --resources=cpus:1;mem:128 --env='{"MY_FILEPATH":
>>> "/home/john/temp_output/test.csv"}'
>>>
>>> john@cluster:~/mesos$ Failed to load flag 'env': Failed to load value
>>> ''{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'': syntax error at
>>> line 1 near: '{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'
>>>
>>>
>>> We're using the Bash shell. Is anyone else passing file-paths within
>>> their environment variable argument?
>>>
>>>
>>> Thanks,
>>> June Taylor
>>> System Administrator, Minnesota Population Center
>>> University of Minnesota
>>>
>>> On Wed, Jun 29, 2016 at 8:55 PM, haosdent  wrote:
>>>
 Yes, need add quotes for --env='{"key": "value"}' flag.

 On Thu, Jun 30, 2016 at 4:39 AM, Chris Baker 
 wrote:

> It's your shell:
>
> $ echo {"MY_FILEPATH": "/home/john/temp_output/test.csv"}
> {MY_FILEPATH: /home/john/temp_output/test.csv}
>
> On Wed, Jun 29, 2016 at 4:05 PM June Taylor  wrote:
>
>> hasodent,
>>
>> I admit I don't see much of a difference in your version. Are you
>> saying that Mesos doesn't escape the values, and that our program may be
>> doing that on the executor end?
>>
>>
>> Thanks,
>> June Taylor
>> System Administrator, Minnesota Population Center
>> University of Minnesota
>>
>> On Wed, Jun 29, 2016 at 2:02 PM, haosdent  wrote:
>>
>>> Hi, @Taylor I use
>>>
>>> ```
>>> ./src/mesos-execute --env='{"MY_FILEPATH":
>>> "/home/john/temp_output/test.csv"}' --master="localhost:5050" 
>>> --command=env
>>> --name="test"
>>> ```
>>>
>>> to test in my env. The output looks correct in my side
>>>
>>> ```
>>> ...
>>> MY_FILEPATH=/home/john/temp_output/test.csv
>>>
>>> MESOS_SANDBOX=/tmp/mesos/slaves/fe6123b9-8757-4015-be4b-fda901247fce-S0/frameworks/fe6123b9-8757-4015-be4b-fda901247fce-/executors/test/runs/fe818890-2a22-4b2b-aab2-816ed0b631d5
>>> ...
>>> ```
>>>
>>> Not sure if your problem caused by your program
>>>
>>> On Thu, Jun 30, 2016 at 1:58 AM, June Taylor  wrote:
>>>
 Greetings,

 We are using mesos-execute and passing an argument that contains a
 Unix filepath, as follows:

 *mesos-execute --master=cluster.example.com:5050
  --name=test-program
 --command=/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
 read --resources=cpus:1;mem:128 --env={"MY_FILEPATH":
 "/home/john/temp_output/test.csv"}*

 The slashes in the file path are then apparently escaped by
 mesos-execute and therefore fail to be useful once picked up by the
 executor:

 *FileNotFoundError: [Errno 2] No such file or directory:
 '"\\/home\\/john\\/temp_output\\/test.csv"'*

 *I0629 10:13:13.814870 14025 exec.cpp:390] Executor asked to
 shutdown*

 Are other people experiencing this? What is the method to avoid it?
 Thanks,
 June Taylor
 System Administrator, Minnesota Population Center
 University of Minnesota

>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Haosdent Huang
>>>
>>
>>


 --
 Best Regards,
 Haosdent Huang

>>>
>>>
>>
>>
>> --
>> Best Regards,
>> Haosdent Huang
>>
>
>

Re: Escaped characters in the 'env' argument passed to mesos-execute

[Erik Weathers]

+1 I would wrap every string in quotes...  otherwise your shell doesn't
know what you mean.  i.e.,

How is the shell supposed to know that you want this all to be 1 string
value for the --command parameter?

--command=/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py read

Similarly how is the shell to know that you *don't* want to end this
command and start a new one when you have a bare semicolon?

--resources=cpus:1;mem:128

- Erik

On Thu, Jun 30, 2016 at 9:08 AM, haosdent  wrote:

> I use bash as well. How about
> ```
> mesos-execute --master=cluster.example.com:5050 --name=test-program
> --command="/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
> read" --resources="cpus:1;mem:128" --env='{"MY_FILEPATH":
> "/home/john/temp_output/test.csv"}'
> ```
>
> On Fri, Jul 1, 2016 at 12:04 AM, June Taylor  wrote:
>
>> hasodent,
>>
>> We've tried your suggestion but it seems like that's not working out:
>>
>> mesos-execute --master=cluster.example.com:5050 --name=test-program
>> --command=/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
>> read --resources=cpus:1;mem:128 --env='{"MY_FILEPATH":
>> "/home/john/temp_output/test.csv"}'
>>
>> john@cluster:~/mesos$ Failed to load flag 'env': Failed to load value
>> ''{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'': syntax error at
>> line 1 near: '{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'
>>
>>
>> We're using the Bash shell. Is anyone else passing file-paths within
>> their environment variable argument?
>>
>>
>> Thanks,
>> June Taylor
>> System Administrator, Minnesota Population Center
>> University of Minnesota
>>
>> On Wed, Jun 29, 2016 at 8:55 PM, haosdent  wrote:
>>
>>> Yes, need add quotes for --env='{"key": "value"}' flag.
>>>
>>> On Thu, Jun 30, 2016 at 4:39 AM, Chris Baker 
>>> wrote:
>>>
 It's your shell:

 $ echo {"MY_FILEPATH": "/home/john/temp_output/test.csv"}
 {MY_FILEPATH: /home/john/temp_output/test.csv}

 On Wed, Jun 29, 2016 at 4:05 PM June Taylor  wrote:

> hasodent,
>
> I admit I don't see much of a difference in your version. Are you
> saying that Mesos doesn't escape the values, and that our program may be
> doing that on the executor end?
>
>
> Thanks,
> June Taylor
> System Administrator, Minnesota Population Center
> University of Minnesota
>
> On Wed, Jun 29, 2016 at 2:02 PM, haosdent  wrote:
>
>> Hi, @Taylor I use
>>
>> ```
>> ./src/mesos-execute --env='{"MY_FILEPATH":
>> "/home/john/temp_output/test.csv"}' --master="localhost:5050" 
>> --command=env
>> --name="test"
>> ```
>>
>> to test in my env. The output looks correct in my side
>>
>> ```
>> ...
>> MY_FILEPATH=/home/john/temp_output/test.csv
>>
>> MESOS_SANDBOX=/tmp/mesos/slaves/fe6123b9-8757-4015-be4b-fda901247fce-S0/frameworks/fe6123b9-8757-4015-be4b-fda901247fce-/executors/test/runs/fe818890-2a22-4b2b-aab2-816ed0b631d5
>> ...
>> ```
>>
>> Not sure if your problem caused by your program
>>
>> On Thu, Jun 30, 2016 at 1:58 AM, June Taylor  wrote:
>>
>>> Greetings,
>>>
>>> We are using mesos-execute and passing an argument that contains a
>>> Unix filepath, as follows:
>>>
>>> *mesos-execute --master=cluster.example.com:5050
>>>  --name=test-program
>>> --command=/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
>>> read --resources=cpus:1;mem:128 --env={"MY_FILEPATH":
>>> "/home/john/temp_output/test.csv"}*
>>>
>>> The slashes in the file path are then apparently escaped by
>>> mesos-execute and therefore fail to be useful once picked up by the
>>> executor:
>>>
>>> *FileNotFoundError: [Errno 2] No such file or directory:
>>> '"\\/home\\/john\\/temp_output\\/test.csv"'*
>>>
>>> *I0629 10:13:13.814870 14025 exec.cpp:390] Executor asked to
>>> shutdown*
>>>
>>> Are other people experiencing this? What is the method to avoid it?
>>> Thanks,
>>> June Taylor
>>> System Administrator, Minnesota Population Center
>>> University of Minnesota
>>>
>>
>>
>>
>> --
>> Best Regards,
>> Haosdent Huang
>>
>
>
>>>
>>>
>>> --
>>> Best Regards,
>>> Haosdent Huang
>>>
>>
>>
>
>
> --
> Best Regards,
> Haosdent Huang
>

Re: Escaped characters in the 'env' argument passed to mesos-execute

[haosdent]

I use bash as well. How about
```
mesos-execute --master=cluster.example.com:5050 --name=test-program
--command="/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
read" --resources="cpus:1;mem:128" --env='{"MY_FILEPATH":
"/home/john/temp_output/test.csv"}'
```

On Fri, Jul 1, 2016 at 12:04 AM, June Taylor  wrote:

> hasodent,
>
> We've tried your suggestion but it seems like that's not working out:
>
> mesos-execute --master=cluster.example.com:5050 --name=test-program
> --command=/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
> read --resources=cpus:1;mem:128 --env='{"MY_FILEPATH":
> "/home/john/temp_output/test.csv"}'
>
> john@cluster:~/mesos$ Failed to load flag 'env': Failed to load value
> ''{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'': syntax error at
> line 1 near: '{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'
>
>
> We're using the Bash shell. Is anyone else passing file-paths within their
> environment variable argument?
>
>
> Thanks,
> June Taylor
> System Administrator, Minnesota Population Center
> University of Minnesota
>
> On Wed, Jun 29, 2016 at 8:55 PM, haosdent  wrote:
>
>> Yes, need add quotes for --env='{"key": "value"}' flag.
>>
>> On Thu, Jun 30, 2016 at 4:39 AM, Chris Baker 
>> wrote:
>>
>>> It's your shell:
>>>
>>> $ echo {"MY_FILEPATH": "/home/john/temp_output/test.csv"}
>>> {MY_FILEPATH: /home/john/temp_output/test.csv}
>>>
>>> On Wed, Jun 29, 2016 at 4:05 PM June Taylor  wrote:
>>>
 hasodent,

 I admit I don't see much of a difference in your version. Are you
 saying that Mesos doesn't escape the values, and that our program may be
 doing that on the executor end?


 Thanks,
 June Taylor
 System Administrator, Minnesota Population Center
 University of Minnesota

 On Wed, Jun 29, 2016 at 2:02 PM, haosdent  wrote:

> Hi, @Taylor I use
>
> ```
> ./src/mesos-execute --env='{"MY_FILEPATH":
> "/home/john/temp_output/test.csv"}' --master="localhost:5050" 
> --command=env
> --name="test"
> ```
>
> to test in my env. The output looks correct in my side
>
> ```
> ...
> MY_FILEPATH=/home/john/temp_output/test.csv
>
> MESOS_SANDBOX=/tmp/mesos/slaves/fe6123b9-8757-4015-be4b-fda901247fce-S0/frameworks/fe6123b9-8757-4015-be4b-fda901247fce-/executors/test/runs/fe818890-2a22-4b2b-aab2-816ed0b631d5
> ...
> ```
>
> Not sure if your problem caused by your program
>
> On Thu, Jun 30, 2016 at 1:58 AM, June Taylor  wrote:
>
>> Greetings,
>>
>> We are using mesos-execute and passing an argument that contains a
>> Unix filepath, as follows:
>>
>> *mesos-execute --master=cluster.example.com:5050
>>  --name=test-program
>> --command=/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
>> read --resources=cpus:1;mem:128 --env={"MY_FILEPATH":
>> "/home/john/temp_output/test.csv"}*
>>
>> The slashes in the file path are then apparently escaped by
>> mesos-execute and therefore fail to be useful once picked up by the
>> executor:
>>
>> *FileNotFoundError: [Errno 2] No such file or directory:
>> '"\\/home\\/john\\/temp_output\\/test.csv"'*
>>
>> *I0629 10:13:13.814870 14025 exec.cpp:390] Executor asked to shutdown*
>>
>> Are other people experiencing this? What is the method to avoid it?
>> Thanks,
>> June Taylor
>> System Administrator, Minnesota Population Center
>> University of Minnesota
>>
>
>
>
> --
> Best Regards,
> Haosdent Huang
>


>>
>>
>> --
>> Best Regards,
>> Haosdent Huang
>>
>
>


-- 
Best Regards,
Haosdent Huang

Re: Escaped characters in the 'env' argument passed to mesos-execute

[June Taylor]

hasodent,

We've tried your suggestion but it seems like that's not working out:

mesos-execute --master=cluster.example.com:5050 --name=test-program
--command=/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
read --resources=cpus:1;mem:128 --env='{"MY_FILEPATH":
"/home/john/temp_output/test.csv"}'

john@cluster:~/mesos$ Failed to load flag 'env': Failed to load value
''{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'': syntax error at
line 1 near: '{"MY_FILEPATH": "/home/john/temp_output/test.csv"}'


We're using the Bash shell. Is anyone else passing file-paths within their
environment variable argument?


Thanks,
June Taylor
System Administrator, Minnesota Population Center
University of Minnesota

On Wed, Jun 29, 2016 at 8:55 PM, haosdent  wrote:

> Yes, need add quotes for --env='{"key": "value"}' flag.
>
> On Thu, Jun 30, 2016 at 4:39 AM, Chris Baker 
> wrote:
>
>> It's your shell:
>>
>> $ echo {"MY_FILEPATH": "/home/john/temp_output/test.csv"}
>> {MY_FILEPATH: /home/john/temp_output/test.csv}
>>
>> On Wed, Jun 29, 2016 at 4:05 PM June Taylor  wrote:
>>
>>> hasodent,
>>>
>>> I admit I don't see much of a difference in your version. Are you saying
>>> that Mesos doesn't escape the values, and that our program may be doing
>>> that on the executor end?
>>>
>>>
>>> Thanks,
>>> June Taylor
>>> System Administrator, Minnesota Population Center
>>> University of Minnesota
>>>
>>> On Wed, Jun 29, 2016 at 2:02 PM, haosdent  wrote:
>>>
 Hi, @Taylor I use

 ```
 ./src/mesos-execute --env='{"MY_FILEPATH":
 "/home/john/temp_output/test.csv"}' --master="localhost:5050" --command=env
 --name="test"
 ```

 to test in my env. The output looks correct in my side

 ```
 ...
 MY_FILEPATH=/home/john/temp_output/test.csv

 MESOS_SANDBOX=/tmp/mesos/slaves/fe6123b9-8757-4015-be4b-fda901247fce-S0/frameworks/fe6123b9-8757-4015-be4b-fda901247fce-/executors/test/runs/fe818890-2a22-4b2b-aab2-816ed0b631d5
 ...
 ```

 Not sure if your problem caused by your program

 On Thu, Jun 30, 2016 at 1:58 AM, June Taylor  wrote:

> Greetings,
>
> We are using mesos-execute and passing an argument that contains a
> Unix filepath, as follows:
>
> *mesos-execute --master=cluster.example.com:5050
>  --name=test-program
> --command=/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
> read --resources=cpus:1;mem:128 --env={"MY_FILEPATH":
> "/home/john/temp_output/test.csv"}*
>
> The slashes in the file path are then apparently escaped by
> mesos-execute and therefore fail to be useful once picked up by the
> executor:
>
> *FileNotFoundError: [Errno 2] No such file or directory:
> '"\\/home\\/john\\/temp_output\\/test.csv"'*
>
> *I0629 10:13:13.814870 14025 exec.cpp:390] Executor asked to shutdown*
>
> Are other people experiencing this? What is the method to avoid it?
> Thanks,
> June Taylor
> System Administrator, Minnesota Population Center
> University of Minnesota
>



 --
 Best Regards,
 Haosdent Huang

>>>
>>>
>
>
> --
> Best Regards,
> Haosdent Huang
>

Re: Escaped characters in the 'env' argument passed to mesos-execute

[haosdent]

Yes, need add quotes for --env='{"key": "value"}' flag.

On Thu, Jun 30, 2016 at 4:39 AM, Chris Baker  wrote:

> It's your shell:
>
> $ echo {"MY_FILEPATH": "/home/john/temp_output/test.csv"}
> {MY_FILEPATH: /home/john/temp_output/test.csv}
>
> On Wed, Jun 29, 2016 at 4:05 PM June Taylor  wrote:
>
>> hasodent,
>>
>> I admit I don't see much of a difference in your version. Are you saying
>> that Mesos doesn't escape the values, and that our program may be doing
>> that on the executor end?
>>
>>
>> Thanks,
>> June Taylor
>> System Administrator, Minnesota Population Center
>> University of Minnesota
>>
>> On Wed, Jun 29, 2016 at 2:02 PM, haosdent  wrote:
>>
>>> Hi, @Taylor I use
>>>
>>> ```
>>> ./src/mesos-execute --env='{"MY_FILEPATH":
>>> "/home/john/temp_output/test.csv"}' --master="localhost:5050" --command=env
>>> --name="test"
>>> ```
>>>
>>> to test in my env. The output looks correct in my side
>>>
>>> ```
>>> ...
>>> MY_FILEPATH=/home/john/temp_output/test.csv
>>>
>>> MESOS_SANDBOX=/tmp/mesos/slaves/fe6123b9-8757-4015-be4b-fda901247fce-S0/frameworks/fe6123b9-8757-4015-be4b-fda901247fce-/executors/test/runs/fe818890-2a22-4b2b-aab2-816ed0b631d5
>>> ...
>>> ```
>>>
>>> Not sure if your problem caused by your program
>>>
>>> On Thu, Jun 30, 2016 at 1:58 AM, June Taylor  wrote:
>>>
 Greetings,

 We are using mesos-execute and passing an argument that contains a Unix
 filepath, as follows:

 *mesos-execute --master=cluster.example.com:5050
  --name=test-program
 --command=/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
 read --resources=cpus:1;mem:128 --env={"MY_FILEPATH":
 "/home/john/temp_output/test.csv"}*

 The slashes in the file path are then apparently escaped by
 mesos-execute and therefore fail to be useful once picked up by the
 executor:

 *FileNotFoundError: [Errno 2] No such file or directory:
 '"\\/home\\/john\\/temp_output\\/test.csv"'*

 *I0629 10:13:13.814870 14025 exec.cpp:390] Executor asked to shutdown*

 Are other people experiencing this? What is the method to avoid it?
 Thanks,
 June Taylor
 System Administrator, Minnesota Population Center
 University of Minnesota

>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Haosdent Huang
>>>
>>
>>


-- 
Best Regards,
Haosdent Huang

Re: Escaped characters in the 'env' argument passed to mesos-execute

[Chris Baker]

It's your shell:

$ echo {"MY_FILEPATH": "/home/john/temp_output/test.csv"}
{MY_FILEPATH: /home/john/temp_output/test.csv}

On Wed, Jun 29, 2016 at 4:05 PM June Taylor  wrote:

> hasodent,
>
> I admit I don't see much of a difference in your version. Are you saying
> that Mesos doesn't escape the values, and that our program may be doing
> that on the executor end?
>
>
> Thanks,
> June Taylor
> System Administrator, Minnesota Population Center
> University of Minnesota
>
> On Wed, Jun 29, 2016 at 2:02 PM, haosdent  wrote:
>
>> Hi, @Taylor I use
>>
>> ```
>> ./src/mesos-execute --env='{"MY_FILEPATH":
>> "/home/john/temp_output/test.csv"}' --master="localhost:5050" --command=env
>> --name="test"
>> ```
>>
>> to test in my env. The output looks correct in my side
>>
>> ```
>> ...
>> MY_FILEPATH=/home/john/temp_output/test.csv
>>
>> MESOS_SANDBOX=/tmp/mesos/slaves/fe6123b9-8757-4015-be4b-fda901247fce-S0/frameworks/fe6123b9-8757-4015-be4b-fda901247fce-/executors/test/runs/fe818890-2a22-4b2b-aab2-816ed0b631d5
>> ...
>> ```
>>
>> Not sure if your problem caused by your program
>>
>> On Thu, Jun 30, 2016 at 1:58 AM, June Taylor  wrote:
>>
>>> Greetings,
>>>
>>> We are using mesos-execute and passing an argument that contains a Unix
>>> filepath, as follows:
>>>
>>> *mesos-execute --master=cluster.example.com:5050
>>>  --name=test-program
>>> --command=/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
>>> read --resources=cpus:1;mem:128 --env={"MY_FILEPATH":
>>> "/home/john/temp_output/test.csv"}*
>>>
>>> The slashes in the file path are then apparently escaped by
>>> mesos-execute and therefore fail to be useful once picked up by the
>>> executor:
>>>
>>> *FileNotFoundError: [Errno 2] No such file or directory:
>>> '"\\/home\\/john\\/temp_output\\/test.csv"'*
>>>
>>> *I0629 10:13:13.814870 14025 exec.cpp:390] Executor asked to shutdown*
>>>
>>> Are other people experiencing this? What is the method to avoid it?
>>> Thanks,
>>> June Taylor
>>> System Administrator, Minnesota Population Center
>>> University of Minnesota
>>>
>>
>>
>>
>> --
>> Best Regards,
>> Haosdent Huang
>>
>
>

Re: Escaped characters in the 'env' argument passed to mesos-execute

[June Taylor]

hasodent,

I admit I don't see much of a difference in your version. Are you saying
that Mesos doesn't escape the values, and that our program may be doing
that on the executor end?


Thanks,
June Taylor
System Administrator, Minnesota Population Center
University of Minnesota

On Wed, Jun 29, 2016 at 2:02 PM, haosdent  wrote:

> Hi, @Taylor I use
>
> ```
> ./src/mesos-execute --env='{"MY_FILEPATH":
> "/home/john/temp_output/test.csv"}' --master="localhost:5050" --command=env
> --name="test"
> ```
>
> to test in my env. The output looks correct in my side
>
> ```
> ...
> MY_FILEPATH=/home/john/temp_output/test.csv
>
> MESOS_SANDBOX=/tmp/mesos/slaves/fe6123b9-8757-4015-be4b-fda901247fce-S0/frameworks/fe6123b9-8757-4015-be4b-fda901247fce-/executors/test/runs/fe818890-2a22-4b2b-aab2-816ed0b631d5
> ...
> ```
>
> Not sure if your problem caused by your program
>
> On Thu, Jun 30, 2016 at 1:58 AM, June Taylor  wrote:
>
>> Greetings,
>>
>> We are using mesos-execute and passing an argument that contains a Unix
>> filepath, as follows:
>>
>> *mesos-execute --master=cluster.example.com:5050
>>  --name=test-program
>> --command=/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
>> read --resources=cpus:1;mem:128 --env={"MY_FILEPATH":
>> "/home/john/temp_output/test.csv"}*
>>
>> The slashes in the file path are then apparently escaped by mesos-execute
>> and therefore fail to be useful once picked up by the executor:
>>
>> *FileNotFoundError: [Errno 2] No such file or directory:
>> '"\\/home\\/john\\/temp_output\\/test.csv"'*
>>
>> *I0629 10:13:13.814870 14025 exec.cpp:390] Executor asked to shutdown*
>>
>> Are other people experiencing this? What is the method to avoid it?
>> Thanks,
>> June Taylor
>> System Administrator, Minnesota Population Center
>> University of Minnesota
>>
>
>
>
> --
> Best Regards,
> Haosdent Huang
>

Escaped characters in the 'env' argument passed to mesos-execute

[June Taylor]

Greetings,

We are using mesos-execute and passing an argument that contains a Unix
filepath, as follows:

*mesos-execute --master=cluster.example.com:5050
 --name=test-program
--command=/home/john/anaconda3/bin/python /home/john/mesos/error_msg.py
read --resources=cpus:1;mem:128 --env={"MY_FILEPATH":
"/home/john/temp_output/test.csv"}*

The slashes in the file path are then apparently escaped by mesos-execute
and therefore fail to be useful once picked up by the executor:

*FileNotFoundError: [Errno 2] No such file or directory:
'"\\/home\\/john\\/temp_output\\/test.csv"'*

*I0629 10:13:13.814870 14025 exec.cpp:390] Executor asked to shutdown*

Are other people experiencing this? What is the method to avoid it?
Thanks,
June Taylor
System Administrator, Minnesota Population Center
University of Minnesota