Re: Review Request 68420: Added regression test for encoded queries over the `/files` API.

2018-08-23 Thread Andrew Schwartzmeyer 


> On Aug. 23, 2018, 1:02 p.m., Benjamin Mahler wrote:
> > src/tests/files_tests.cpp
> > Lines 326 (patched)
> > 
> >
> > How about: `ReservedQueryCharacter`

It's not quite the same. This isn't testing any arbitray reserved character 
existing in the query; it's explicitly testing a percent-encoded sequence in 
the query. How about: `QueryWithEncodedSequence`?


> On Aug. 23, 2018, 1:02 p.m., Benjamin Mahler wrote:
> > src/tests/files_tests.cpp
> > Lines 331-332 (patched)
> > 
> >
> > This is a little odd to read without the context, this tests reserved 
> > characters by using `%`, but we could use anything other reserved character 
> > to acheive the same test, e.g. `+` seems the simplest:
> > 
> > ```
> >   // We use a reserved character for query parameters `+` to
> >   // ensure it is encoded and decoded correctly.
> >   const string filename = "foo+bar";
> > ```
> > 
> > If you'd like to stick with using `%`, probably we should just say:
> > 
> > ```
> >   // We use a reserved character for query parameters `%` to
> >   // ensure it is encoded and decoded correctly.
> >   const string filename = "foo%3Abar";
> > ```
> > 
> > And not do line 334 as it's irrelevant to this test

You couldn't use `+` because it wouldn't have the same problem: the second 
decode would not turn `+` into something else (this is why we haven't run into 
this problem before with other characters, even a literal `:`). The problem is 
that `%3A` _can_ be decoded, and so decoding more times than encoding causes it 
to erroneously become `:`. The number of encodes and decodes must be balanced.

Also, line 334 is relevant to this test: it's demonstrating that the query can 
be erroneously decoded too many times.


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68420/#review207830
---


On Aug. 17, 2018, 6:20 p.m., Andrew Schwartzmeyer wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68420/
> ---
> 
> (Updated Aug. 17, 2018, 6:20 p.m.)
> 
> 
> Review request for mesos and Benjamin Mahler.
> 
> 
> Bugs: MESOS-9168
> https://issues.apache.org/jira/browse/MESOS-9168
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This test currently fails with:
> 
> ```
> src\tests\files_tests.cpp(346): error: Value of: (response)->status
>   Actual: "404 Not Found"
> Expected: OK().status
> Which is: "200 OK"
> Body: ""
> src\tests\files_tests.cpp(347): error: Value of: (response)->body
>   Actual: ""
> Expected: stringify(expected)
> Which is: "{\"data\":\"body\"}"
> ```
> 
> This is due to libprocess decoding the query in both `http::get()` and
> in the `DataDecoder`. However, libprocess only encodes once. When the
> user has a query with a literal ASCII escape sequence, it is expected
> that it should only need to be encoded once. That is `foo%3Abar`
> should be represented in the query as `foo%253Abar`. Because
> libprocess decodes twice, but currently only encodes once, this query
> erroneously becomes `foo:bar`. The added test demonstrates how this
> breaks something such as the `files` server.
> 
> 
> Diffs
> -
> 
>   src/tests/files_tests.cpp d09279077ec704167991933349ee943ab44d0aa9 
> 
> 
> Diff: https://reviews.apache.org/r/68420/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Andrew Schwartzmeyer
> 
>

Re: Review Request 68420: Added regression test for encoded queries over the `/files` API.

2018-08-23 Thread Benjamin Mahler 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68420/#review207830
---



How about the following for commit summary and description:

```
Added /files API test for reserved query characters.

Due to a bug in the http client in libprocess (MESOS-9168), the
use of reserved query characters in paths did not work when using
the http client in libprocess. This adds a test to ensure that
the /files API correctly handles reserved query characters in
file paths.
```


src/tests/files_tests.cpp
Lines 323-325 (patched)


How about:

```
// Tests paths with reserved characters that must be percent
// encoded in the http request query.
```



src/tests/files_tests.cpp
Lines 326 (patched)


How about: `ReservedQueryCharacter`



src/tests/files_tests.cpp
Lines 331-332 (patched)


This is a little odd to read without the context, this tests reserved 
characters by using `%`, but we could use anything other reserved character to 
acheive the same test, e.g. `+` seems the simplest:

```
  // We use a reserved character for query parameters `+` to
  // ensure it is encoded and decoded correctly.
  const string filename = "foo+bar";
```

If you'd like to stick with using `%`, probably we should just say:

```
  // We use a reserved character for query parameters `%` to
  // ensure it is encoded and decoded correctly.
  const string filename = "foo%3Abar";
```

And not do line 334 as it's irrelevant to this test


- Benjamin Mahler


On Aug. 18, 2018, 1:20 a.m., Andrew Schwartzmeyer wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68420/
> ---
> 
> (Updated Aug. 18, 2018, 1:20 a.m.)
> 
> 
> Review request for mesos and Benjamin Mahler.
> 
> 
> Bugs: MESOS-9168
> https://issues.apache.org/jira/browse/MESOS-9168
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This test currently fails with:
> 
> ```
> src\tests\files_tests.cpp(346): error: Value of: (response)->status
>   Actual: "404 Not Found"
> Expected: OK().status
> Which is: "200 OK"
> Body: ""
> src\tests\files_tests.cpp(347): error: Value of: (response)->body
>   Actual: ""
> Expected: stringify(expected)
> Which is: "{\"data\":\"body\"}"
> ```
> 
> This is due to libprocess decoding the query in both `http::get()` and
> in the `DataDecoder`. However, libprocess only encodes once. When the
> user has a query with a literal ASCII escape sequence, it is expected
> that it should only need to be encoded once. That is `foo%3Abar`
> should be represented in the query as `foo%253Abar`. Because
> libprocess decodes twice, but currently only encodes once, this query
> erroneously becomes `foo:bar`. The added test demonstrates how this
> breaks something such as the `files` server.
> 
> 
> Diffs
> -
> 
>   src/tests/files_tests.cpp d09279077ec704167991933349ee943ab44d0aa9 
> 
> 
> Diff: https://reviews.apache.org/r/68420/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Andrew Schwartzmeyer
> 
>

Review Request 68420: Added regression test for encoded queries over the `/files` API.

2018-08-17 Thread Andrew Schwartzmeyer 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68420/
---

Review request for mesos and Benjamin Mahler.


Repository: mesos


Description
---

This test currently fails with:

```
src\tests\files_tests.cpp(346): error: Value of: (response)->status
  Actual: "404 Not Found"
Expected: OK().status
Which is: "200 OK"
Body: ""
src\tests\files_tests.cpp(347): error: Value of: (response)->body
  Actual: ""
Expected: stringify(expected)
Which is: "{\"data\":\"body\"}"
```

This is due to libprocess decoding the query in both `http::get()` and
in the `DataDecoder`. However, libprocess only encodes once. When the
user has a query with a literal ASCII escape sequence, it is expected
that it should only need to be encoded once. That is `foo%3Abar`
should be represented in the query as `foo%253Abar`. Because
libprocess decodes twice, but currently only encodes once, this query
erroneously becomes `foo:bar`. The added test demonstrates how this
breaks something such as the `files` server.


Diffs
-

  src/tests/files_tests.cpp d09279077ec704167991933349ee943ab44d0aa9 


Diff: https://reviews.apache.org/r/68420/diff/1/


Testing
---


Thanks,

Andrew Schwartzmeyer