Re: Review Request 66424: Windows: Replaced `_wopen()` with `CreateFileW()` in `os::open()`.

2018-04-20 Thread Andrew Schwartzmeyer 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66424/
---

(Updated April 20, 2018, 11:44 a.m.)


Review request for mesos, Akash Gupta, Eric Mumau, John Kordich, Joseph Wu, and 
Michael Park.


Changes
---

Removed old comment.


Bugs: MESOS-8673
https://issues.apache.org/jira/browse/MESOS-8673


Repository: mesos


Description
---

Instead of using the CRT implementation of `_wopen()` for the
`os::open()` API, we now use the Windows API `CreateFileW()`, mapping
each of the Linux `open()` flags to their semantic equivalents. This
will make implementing overlapped I/O possible, and is a step toward
removing the use of integer file descriptors on Windows.

Note that instead of redefining the C flags like `O_RDONLY`, we just
use them directly in our mapping logic, and set the used but
unsupported flags to zero.

This change uncovered several bugs such as incorrect access flags, and
used-but-not-included headers.

We currently ignore creation permissions as they will be handled in a
broader project to map permissions to Windows correctly.


Diffs (updated)
-

  3rdparty/stout/include/stout/net.hpp d2992c05b221ea90dae1c06d27753932f7411925 
  3rdparty/stout/include/stout/os/windows/fcntl.hpp 
bf8c38acad60f9b0eb752053dcd53a9fda7b8bfa 
  3rdparty/stout/include/stout/os/windows/mktemp.hpp 
5c775c45c415d9ddd6a80ab814fb55475e9f871e 
  3rdparty/stout/include/stout/os/windows/open.hpp PRE-CREATION 
  3rdparty/stout/include/stout/windows.hpp 
1bfcdf4a5c097cc6d2293396ce39c8ad2c9ec993 


Diff: https://reviews.apache.org/r/66424/diff/3/

Changes: https://reviews.apache.org/r/66424/diff/2-3/


Testing
---


Thanks,

Andrew Schwartzmeyer

Re: Review Request 66424: Windows: Replaced `_wopen()` with `CreateFileW()` in `os::open()`.

2018-04-18 Thread Joseph Wu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66424/#review201021
---


Fix it, then Ship it!





3rdparty/stout/include/stout/os/windows/mktemp.hpp
Lines 62-64 (original), 62-65 (patched)


The last sentence of this comment can be updated a bit to match.



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 29-31 (original), 26-28 (patched)


Hm... This review ends up deleting the definitions in `fcntl.hpp`.

Seems like this comment change was intended for the previous review ( 
https://reviews.apache.org/r/66423 )?


- Joseph Wu


On April 6, 2018, 3:52 p.m., Andrew Schwartzmeyer wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66424/
> ---
> 
> (Updated April 6, 2018, 3:52 p.m.)
> 
> 
> Review request for mesos, Akash Gupta, Eric Mumau, John Kordich, Joseph Wu, 
> and Michael Park.
> 
> 
> Bugs: MESOS-8673
> https://issues.apache.org/jira/browse/MESOS-8673
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Instead of using the CRT implementation of `_wopen()` for the
> `os::open()` API, we now use the Windows API `CreateFileW()`, mapping
> each of the Linux `open()` flags to their semantic equivalents. This
> will make implementing overlapped I/O possible, and is a step toward
> removing the use of integer file descriptors on Windows.
> 
> Note that instead of redefining the C flags like `O_RDONLY`, we just
> use them directly in our mapping logic, and set the used but
> unsupported flags to zero.
> 
> This change uncovered several bugs such as incorrect access flags, and
> used-but-not-included headers.
> 
> We currently ignore creation permissions as they will be handled in a
> broader project to map permissions to Windows correctly.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/net.hpp 
> d2992c05b221ea90dae1c06d27753932f7411925 
>   3rdparty/stout/include/stout/os/windows/fcntl.hpp 
> bf8c38acad60f9b0eb752053dcd53a9fda7b8bfa 
>   3rdparty/stout/include/stout/os/windows/mktemp.hpp 
> 5c775c45c415d9ddd6a80ab814fb55475e9f871e 
>   3rdparty/stout/include/stout/os/windows/open.hpp PRE-CREATION 
>   3rdparty/stout/include/stout/windows.hpp 
> 1bfcdf4a5c097cc6d2293396ce39c8ad2c9ec993 
> 
> 
> Diff: https://reviews.apache.org/r/66424/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Andrew Schwartzmeyer
> 
>

Re: Review Request 66424: Windows: Replaced `_wopen()` with `CreateFileW()` in `os::open()`.

2018-04-09 Thread Akash Gupta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66424/#review200749
---


Ship it!




Ship It!

- Akash Gupta


On April 6, 2018, 10:52 p.m., Andrew Schwartzmeyer wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66424/
> ---
> 
> (Updated April 6, 2018, 10:52 p.m.)
> 
> 
> Review request for mesos, Akash Gupta, Eric Mumau, John Kordich, Joseph Wu, 
> and Michael Park.
> 
> 
> Bugs: MESOS-8673
> https://issues.apache.org/jira/browse/MESOS-8673
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Instead of using the CRT implementation of `_wopen()` for the
> `os::open()` API, we now use the Windows API `CreateFileW()`, mapping
> each of the Linux `open()` flags to their semantic equivalents. This
> will make implementing overlapped I/O possible, and is a step toward
> removing the use of integer file descriptors on Windows.
> 
> Note that instead of redefining the C flags like `O_RDONLY`, we just
> use them directly in our mapping logic, and set the used but
> unsupported flags to zero.
> 
> This change uncovered several bugs such as incorrect access flags, and
> used-but-not-included headers.
> 
> We currently ignore creation permissions as they will be handled in a
> broader project to map permissions to Windows correctly.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/net.hpp 
> d2992c05b221ea90dae1c06d27753932f7411925 
>   3rdparty/stout/include/stout/os/windows/fcntl.hpp 
> bf8c38acad60f9b0eb752053dcd53a9fda7b8bfa 
>   3rdparty/stout/include/stout/os/windows/mktemp.hpp 
> 5c775c45c415d9ddd6a80ab814fb55475e9f871e 
>   3rdparty/stout/include/stout/os/windows/open.hpp PRE-CREATION 
>   3rdparty/stout/include/stout/windows.hpp 
> 1bfcdf4a5c097cc6d2293396ce39c8ad2c9ec993 
> 
> 
> Diff: https://reviews.apache.org/r/66424/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Andrew Schwartzmeyer
> 
>

Re: Review Request 66424: Windows: Replaced `_wopen()` with `CreateFileW()` in `os::open()`.

2018-04-06 Thread Andrew Schwartzmeyer 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66424/
---

(Updated April 6, 2018, 3:52 p.m.)


Review request for mesos, Akash Gupta, Eric Mumau, John Kordich, Joseph Wu, and 
Michael Park.


Changes
---

Easier to follow logic with `switch` statements.


Bugs: MESOS-8673
https://issues.apache.org/jira/browse/MESOS-8673


Repository: mesos


Description
---

Instead of using the CRT implementation of `_wopen()` for the
`os::open()` API, we now use the Windows API `CreateFileW()`, mapping
each of the Linux `open()` flags to their semantic equivalents. This
will make implementing overlapped I/O possible, and is a step toward
removing the use of integer file descriptors on Windows.

Note that instead of redefining the C flags like `O_RDONLY`, we just
use them directly in our mapping logic, and set the used but
unsupported flags to zero.

This change uncovered several bugs such as incorrect access flags, and
used-but-not-included headers.

We currently ignore creation permissions as they will be handled in a
broader project to map permissions to Windows correctly.


Diffs (updated)
-

  3rdparty/stout/include/stout/net.hpp d2992c05b221ea90dae1c06d27753932f7411925 
  3rdparty/stout/include/stout/os/windows/fcntl.hpp 
bf8c38acad60f9b0eb752053dcd53a9fda7b8bfa 
  3rdparty/stout/include/stout/os/windows/mktemp.hpp 
5c775c45c415d9ddd6a80ab814fb55475e9f871e 
  3rdparty/stout/include/stout/os/windows/open.hpp PRE-CREATION 
  3rdparty/stout/include/stout/windows.hpp 
1bfcdf4a5c097cc6d2293396ce39c8ad2c9ec993 


Diff: https://reviews.apache.org/r/66424/diff/2/

Changes: https://reviews.apache.org/r/66424/diff/1-2/


Testing
---


Thanks,

Andrew Schwartzmeyer

Re: Review Request 66424: Windows: Replaced `_wopen()` with `CreateFileW()` in `os::open()`.

2018-04-05 Thread Andrew Schwartzmeyer 


> On April 5, 2018, 4:07 p.m., Akash Gupta wrote:
> > 3rdparty/stout/include/stout/os/windows/mktemp.hpp
> > Line 54 (original), 54 (patched)
> > 
> >
> > Huh that's a strange function. Seems like it can cause time check/use 
> > race conditions. We probably want to replace this with the native temp 
> > directory functions in the future.

Ha, I agree. It's listed in MESOS-8682 so it'll get done eventually.


> On April 5, 2018, 4:07 p.m., Akash Gupta wrote:
> > 3rdparty/stout/include/stout/os/windows/open.hpp
> > Lines 37 (patched)
> > 
> >
> > I *think* FILE_FLAG_WRITE_THROUGH == O_SYNC.

It's... similar, but I looked at the use-cases and I think we're more likely to 
introduce unintended behaior by trying to guess what the user wants here.


> On April 5, 2018, 4:07 p.m., Akash Gupta wrote:
> > 3rdparty/stout/include/stout/os/windows/open.hpp
> > Lines 45 (patched)
> > 
> >
> > I believe the default behavior(`SECURITY_ATTRIBUTES == NULL`) is to 
> > inherit from the parent directory / access token. We should ask around on 
> > what's the proper settings for security.

Don't let me forget to file/update/mention the permissions issue.


> On April 5, 2018, 4:07 p.m., Akash Gupta wrote:
> > 3rdparty/stout/include/stout/os/windows/open.hpp
> > Lines 61 (patched)
> > 
> >
> > It's up to you, but I find it clearer to have a switch statement on the 
> > bitfields like this:
> > ```
> > switch(oflag & (O_RDONLY | O_WRONLY | O_RDWR)) {
> >   case O_RDONLY:
> >   case O_WRONLY:
> >   case O_RDWR:
> >   default:
> > }
> > ```

Only problem is that O_RDONLY is 0, so case O_RDONLY will never execute. Some 
brilliant programmer somewhere though that 0 would work great as a bit flag.


> On April 5, 2018, 4:07 p.m., Akash Gupta wrote:
> > 3rdparty/stout/include/stout/os/windows/open.hpp
> > Lines 73 (patched)
> > 
> >
> > Same thing as above. I think it's more import on this one because 
> > there's techically 8 cases for `O_CREAT`, `O_EXCL`, `O_TRUNC`, but POSIX 
> > and Windows only have defined behavior for 5 of them, so you can explicitly 
> > mention the undefined behavior cases like this:
> > 
> > ```
> > switch(oflag & (O_CREAT | O_EXCL | O_TRUNC)) {
> >   case O_CREAT | O_EXCL:
> >   case O_CREAT | O_EXCL | O_TRUNC :
> >// Ignore O_TRUNC if we get O_CREAT | O_EXCL
> >create = CREATE_NEW; break;
> >   case ...:
> > }

Yeah I was not a huge fan of these if/else chains. Thanks.


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66424/#review200579
---


On April 3, 2018, 10:47 p.m., Andrew Schwartzmeyer wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66424/
> ---
> 
> (Updated April 3, 2018, 10:47 p.m.)
> 
> 
> Review request for mesos, Akash Gupta, Eric Mumau, John Kordich, Joseph Wu, 
> and Michael Park.
> 
> 
> Bugs: MESOS-8673
> https://issues.apache.org/jira/browse/MESOS-8673
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Instead of using the CRT implementation of `_wopen()` for the
> `os::open()` API, we now use the Windows API `CreateFileW()`, mapping
> each of the Linux `open()` flags to their semantic equivalents. This
> will make implementing overlapped I/O possible, and is a step toward
> removing the use of integer file descriptors on Windows.
> 
> Note that instead of redefining the C flags like `O_RDONLY`, we just
> use them directly in our mapping logic, and set the used but
> unsupported flags to zero.
> 
> This change uncovered several bugs such as incorrect access flags, and
> used-but-not-included headers.
> 
> We currently ignore creation permissions as they will be handled in a
> broader project to map permissions to Windows correctly.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/net.hpp 
> d2992c05b221ea90dae1c06d27753932f7411925 
>   3rdparty/stout/include/stout/os/windows/fcntl.hpp 
> bf8c38acad60f9b0eb752053dcd53a9fda7b8bfa 
>   3rdparty/stout/include/stout/os/windows/mktemp.hpp 
> 5c775c45c415d9ddd6a80ab814fb55475e9f871e 
>   3rdparty/stout/include/stout/os/windows/open.hpp PRE-CREATION 
>   3rdparty/stout/include/stout/windows.hpp 
> 1bfcdf4a5c097cc6d2293396ce39c8ad2c9ec993 
> 
> 
> Diff: https://reviews.apache.org/r/66424/diff/1/
> 
> 
> Testing
> ---
>

Re: Review Request 66424: Windows: Replaced `_wopen()` with `CreateFileW()` in `os::open()`.

2018-04-05 Thread Andrew Schwartzmeyer 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66424/#review200591
---




3rdparty/stout/include/stout/os/windows/open.hpp
Lines 37 (patched)


It's... similar, but I looked at the use-cases and I think we're more 
likely to introduce unintended behaior by trying to guess what the user wants 
here.



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 61 (patched)


Only problem is that `O_RDONLY` is 0, so `case O_RDONLY` will never 
execute. Some brilliant programmer somewhere though that `0` would work great 
as a bit flag.



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 73 (patched)


Yeah I was not a huge fan of these if/else chains. Thanks.


- Andrew Schwartzmeyer


On April 3, 2018, 10:47 p.m., Andrew Schwartzmeyer wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66424/
> ---
> 
> (Updated April 3, 2018, 10:47 p.m.)
> 
> 
> Review request for mesos, Akash Gupta, Eric Mumau, John Kordich, Joseph Wu, 
> and Michael Park.
> 
> 
> Bugs: MESOS-8673
> https://issues.apache.org/jira/browse/MESOS-8673
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Instead of using the CRT implementation of `_wopen()` for the
> `os::open()` API, we now use the Windows API `CreateFileW()`, mapping
> each of the Linux `open()` flags to their semantic equivalents. This
> will make implementing overlapped I/O possible, and is a step toward
> removing the use of integer file descriptors on Windows.
> 
> Note that instead of redefining the C flags like `O_RDONLY`, we just
> use them directly in our mapping logic, and set the used but
> unsupported flags to zero.
> 
> This change uncovered several bugs such as incorrect access flags, and
> used-but-not-included headers.
> 
> We currently ignore creation permissions as they will be handled in a
> broader project to map permissions to Windows correctly.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/net.hpp 
> d2992c05b221ea90dae1c06d27753932f7411925 
>   3rdparty/stout/include/stout/os/windows/fcntl.hpp 
> bf8c38acad60f9b0eb752053dcd53a9fda7b8bfa 
>   3rdparty/stout/include/stout/os/windows/mktemp.hpp 
> 5c775c45c415d9ddd6a80ab814fb55475e9f871e 
>   3rdparty/stout/include/stout/os/windows/open.hpp PRE-CREATION 
>   3rdparty/stout/include/stout/windows.hpp 
> 1bfcdf4a5c097cc6d2293396ce39c8ad2c9ec993 
> 
> 
> Diff: https://reviews.apache.org/r/66424/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Andrew Schwartzmeyer
> 
>

Re: Review Request 66424: Windows: Replaced `_wopen()` with `CreateFileW()` in `os::open()`.

2018-04-05 Thread Akash Gupta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66424/#review200579
---


Fix it, then Ship it!





3rdparty/stout/include/stout/os/windows/mktemp.hpp
Line 54 (original), 54 (patched)


Huh that's a strange function. Seems like it can cause time check/use race 
conditions. We probably want to replace this with the native temp directory 
functions in the future.



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 33 (patched)


:)



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 37 (patched)


I *think* FILE_FLAG_WRITE_THROUGH == O_SYNC.



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 45 (patched)


I believe the default behavior(`SECURITY_ATTRIBUTES == NULL`) is to inherit 
from the parent directory / access token. We should ask around on what's the 
proper settings for security.



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 61 (patched)


It's up to you, but I find it clearer to have a switch statement on the 
bitfields like this:
```
switch(oflag & (O_RDONLY | O_WRONLY | O_RDWR)) {
  case O_RDONLY:
  case O_WRONLY:
  case O_RDWR:
  default:
}
```



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 73 (patched)


Same thing as above. I think it's more import on this one because there's 
techically 8 cases for `O_CREAT`, `O_EXCL`, `O_TRUNC`, but POSIX and Windows 
only have defined behavior for 5 of them, so you can explicitly mention the 
undefined behavior cases like this:

```
switch(oflag & (O_CREAT | O_EXCL | O_TRUNC)) {
  case O_CREAT | O_EXCL:
  case O_CREAT | O_EXCL | O_TRUNC :
   // Ignore O_TRUNC if we get O_CREAT | O_EXCL
   create = CREATE_NEW; break;
  case ...:
}


- Akash Gupta


On April 4, 2018, 5:47 a.m., Andrew Schwartzmeyer wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66424/
> ---
> 
> (Updated April 4, 2018, 5:47 a.m.)
> 
> 
> Review request for mesos, Akash Gupta, Eric Mumau, John Kordich, Joseph Wu, 
> and Michael Park.
> 
> 
> Bugs: MESOS-8673
> https://issues.apache.org/jira/browse/MESOS-8673
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Instead of using the CRT implementation of `_wopen()` for the
> `os::open()` API, we now use the Windows API `CreateFileW()`, mapping
> each of the Linux `open()` flags to their semantic equivalents. This
> will make implementing overlapped I/O possible, and is a step toward
> removing the use of integer file descriptors on Windows.
> 
> Note that instead of redefining the C flags like `O_RDONLY`, we just
> use them directly in our mapping logic, and set the used but
> unsupported flags to zero.
> 
> This change uncovered several bugs such as incorrect access flags, and
> used-but-not-included headers.
> 
> We currently ignore creation permissions as they will be handled in a
> broader project to map permissions to Windows correctly.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/net.hpp 
> d2992c05b221ea90dae1c06d27753932f7411925 
>   3rdparty/stout/include/stout/os/windows/fcntl.hpp 
> bf8c38acad60f9b0eb752053dcd53a9fda7b8bfa 
>   3rdparty/stout/include/stout/os/windows/mktemp.hpp 
> 5c775c45c415d9ddd6a80ab814fb55475e9f871e 
>   3rdparty/stout/include/stout/os/windows/open.hpp PRE-CREATION 
>   3rdparty/stout/include/stout/windows.hpp 
> 1bfcdf4a5c097cc6d2293396ce39c8ad2c9ec993 
> 
> 
> Diff: https://reviews.apache.org/r/66424/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Andrew Schwartzmeyer
> 
>

Re: Review Request 66424: Windows: Replaced `_wopen()` with `CreateFileW()` in `os::open()`.

2018-04-05 Thread Akash Gupta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66424/#review200579
---


Fix it, then Ship it!





3rdparty/stout/include/stout/os/windows/mktemp.hpp
Line 54 (original), 54 (patched)


Huh that's a strange function. Seems like it can cause time check/use race 
conditions. We probably want to replace this with the native temp directory 
functions in the future.



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 33 (patched)


:)



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 37 (patched)


I *think* FILE_FLAG_WRITE_THROUGH == O_SYNC.



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 45 (patched)


I believe the default behavior(`SECURITY_ATTRIBUTES == NULL`) is to inherit 
from the parent directory / access token. We should ask around on what's the 
proper settings for security.



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 61 (patched)


It's up to you, but I find it clearer to have a switch statement on the 
bitfields like this:
```
switch(oflag & (O_RDONLY | O_WRONLY | O_RDWR)) {
  case O_RDONLY:
  case O_WRONLY:
  case O_RDWR:
  default:
}
```



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 73 (patched)


Same thing as above. I think it's more import on this one because there's 
techically 8 cases for `O_CREAT`, `O_EXCL`, `O_TRUNC`, but POSIX and Windows 
only have defined behavior for 5 of them, so you can explicitly mention the 
undefined behavior cases like this:

```
switch(oflag & (O_CREAT | O_EXCL | O_TRUNC)) {
  case O_CREAT | O_EXCL:
  case O_CREAT | O_EXCL | O_TRUNC :
   // Ignore O_TRUNC if we get O_CREAT | O_EXCL
   create = CREATE_NEW; break;
  case ...:
}


- Akash Gupta


On April 4, 2018, 5:47 a.m., Andrew Schwartzmeyer wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66424/
> ---
> 
> (Updated April 4, 2018, 5:47 a.m.)
> 
> 
> Review request for mesos, Akash Gupta, Eric Mumau, John Kordich, Joseph Wu, 
> and Michael Park.
> 
> 
> Bugs: MESOS-8673
> https://issues.apache.org/jira/browse/MESOS-8673
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Instead of using the CRT implementation of `_wopen()` for the
> `os::open()` API, we now use the Windows API `CreateFileW()`, mapping
> each of the Linux `open()` flags to their semantic equivalents. This
> will make implementing overlapped I/O possible, and is a step toward
> removing the use of integer file descriptors on Windows.
> 
> Note that instead of redefining the C flags like `O_RDONLY`, we just
> use them directly in our mapping logic, and set the used but
> unsupported flags to zero.
> 
> This change uncovered several bugs such as incorrect access flags, and
> used-but-not-included headers.
> 
> We currently ignore creation permissions as they will be handled in a
> broader project to map permissions to Windows correctly.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/net.hpp 
> d2992c05b221ea90dae1c06d27753932f7411925 
>   3rdparty/stout/include/stout/os/windows/fcntl.hpp 
> bf8c38acad60f9b0eb752053dcd53a9fda7b8bfa 
>   3rdparty/stout/include/stout/os/windows/mktemp.hpp 
> 5c775c45c415d9ddd6a80ab814fb55475e9f871e 
>   3rdparty/stout/include/stout/os/windows/open.hpp PRE-CREATION 
>   3rdparty/stout/include/stout/windows.hpp 
> 1bfcdf4a5c097cc6d2293396ce39c8ad2c9ec993 
> 
> 
> Diff: https://reviews.apache.org/r/66424/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Andrew Schwartzmeyer
> 
>

Re: Review Request 66424: Windows: Replaced `_wopen()` with `CreateFileW()` in `os::open()`.

2018-04-05 Thread Akash Gupta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66424/#review200579
---


Fix it, then Ship it!





3rdparty/stout/include/stout/os/windows/mktemp.hpp
Line 54 (original), 54 (patched)


Huh that's a strange function. Seems like it can cause time check/use race 
conditions. We probably want to replace this with the native temp directory 
functions in the future.



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 33 (patched)


:)



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 37 (patched)


I *think* FILE_FLAG_WRITE_THROUGH == O_SYNC.



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 45 (patched)


I believe the default behavior(`SECURITY_ATTRIBUTES == NULL`) is to inherit 
from the parent directory / access token. We should ask around on what's the 
proper settings for security.



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 61 (patched)


It's up to you, but I find it clearer to have a switch statement on the 
bitfields like this:
```
switch(oflag & (O_RDONLY | O_WRONLY | O_RDWR)) {
  case O_RDONLY:
  case O_WRONLY:
  case O_RDWR:
  default:
}
```



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 73 (patched)


Same thing as above. I think it's more import on this one because there's 
techically 8 cases for `O_CREAT`, `O_EXCL`, `O_TRUNC`, but POSIX and Windows 
only have defined behavior for 5 of them, so you can explicitly mention the 
undefined behavior cases like this:

```
switch(oflag & (O_CREAT | O_EXCL | O_TRUNC)) {
  case O_CREAT | O_EXCL:
  case O_CREAT | O_EXCL | O_TRUNC :
   // Ignore O_TRUNC if we get O_CREAT | O_EXCL
   create = CREATE_NEW; break;
  case ...:
}


- Akash Gupta


On April 4, 2018, 5:47 a.m., Andrew Schwartzmeyer wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66424/
> ---
> 
> (Updated April 4, 2018, 5:47 a.m.)
> 
> 
> Review request for mesos, Akash Gupta, Eric Mumau, John Kordich, Joseph Wu, 
> and Michael Park.
> 
> 
> Bugs: MESOS-8673
> https://issues.apache.org/jira/browse/MESOS-8673
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Instead of using the CRT implementation of `_wopen()` for the
> `os::open()` API, we now use the Windows API `CreateFileW()`, mapping
> each of the Linux `open()` flags to their semantic equivalents. This
> will make implementing overlapped I/O possible, and is a step toward
> removing the use of integer file descriptors on Windows.
> 
> Note that instead of redefining the C flags like `O_RDONLY`, we just
> use them directly in our mapping logic, and set the used but
> unsupported flags to zero.
> 
> This change uncovered several bugs such as incorrect access flags, and
> used-but-not-included headers.
> 
> We currently ignore creation permissions as they will be handled in a
> broader project to map permissions to Windows correctly.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/net.hpp 
> d2992c05b221ea90dae1c06d27753932f7411925 
>   3rdparty/stout/include/stout/os/windows/fcntl.hpp 
> bf8c38acad60f9b0eb752053dcd53a9fda7b8bfa 
>   3rdparty/stout/include/stout/os/windows/mktemp.hpp 
> 5c775c45c415d9ddd6a80ab814fb55475e9f871e 
>   3rdparty/stout/include/stout/os/windows/open.hpp PRE-CREATION 
>   3rdparty/stout/include/stout/windows.hpp 
> 1bfcdf4a5c097cc6d2293396ce39c8ad2c9ec993 
> 
> 
> Diff: https://reviews.apache.org/r/66424/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Andrew Schwartzmeyer
> 
>

Re: Review Request 66424: Windows: Replaced `_wopen()` with `CreateFileW()` in `os::open()`.

2018-04-05 Thread Akash Gupta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66424/#review200579
---


Fix it, then Ship it!





3rdparty/stout/include/stout/os/windows/mktemp.hpp
Line 54 (original), 54 (patched)


Huh that's a strange function. Seems like it can cause time check/use race 
conditions. We probably want to replace this with the native temp directory 
functions in the future.



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 33 (patched)


:)



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 37 (patched)


I *think* FILE_FLAG_WRITE_THROUGH == O_SYNC.



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 45 (patched)


I believe the default behavior(`SECURITY_ATTRIBUTES == NULL`) is to inherit 
from the parent directory / access token. We should ask around on what's the 
proper settings for security.



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 61 (patched)


It's up to you, but I find it clearer to have a switch statement on the 
bitfields like this:
```
switch(oflag & (O_RDONLY | O_WRONLY | O_RDWR)) {
  case O_RDONLY:
  case O_WRONLY:
  case O_RDWR:
  default:
}
```



3rdparty/stout/include/stout/os/windows/open.hpp
Lines 73 (patched)


Same thing as above. I think it's more import on this one because there's 
techically 8 cases for `O_CREAT`, `O_EXCL`, `O_TRUNC`, but POSIX and Windows 
only have defined behavior for 5 of them, so you can explicitly mention the 
undefined behavior cases like this:

```
switch(oflag & (O_CREAT | O_EXCL | O_TRUNC)) {
  case O_CREAT | O_EXCL:
  case O_CREAT | O_EXCL | O_TRUNC :
   // Ignore O_TRUNC if we get O_CREAT | O_EXCL
   create = CREATE_NEW; break;
  case ...:
}


- Akash Gupta


On April 4, 2018, 5:47 a.m., Andrew Schwartzmeyer wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66424/
> ---
> 
> (Updated April 4, 2018, 5:47 a.m.)
> 
> 
> Review request for mesos, Akash Gupta, Eric Mumau, John Kordich, Joseph Wu, 
> and Michael Park.
> 
> 
> Bugs: MESOS-8673
> https://issues.apache.org/jira/browse/MESOS-8673
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Instead of using the CRT implementation of `_wopen()` for the
> `os::open()` API, we now use the Windows API `CreateFileW()`, mapping
> each of the Linux `open()` flags to their semantic equivalents. This
> will make implementing overlapped I/O possible, and is a step toward
> removing the use of integer file descriptors on Windows.
> 
> Note that instead of redefining the C flags like `O_RDONLY`, we just
> use them directly in our mapping logic, and set the used but
> unsupported flags to zero.
> 
> This change uncovered several bugs such as incorrect access flags, and
> used-but-not-included headers.
> 
> We currently ignore creation permissions as they will be handled in a
> broader project to map permissions to Windows correctly.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/net.hpp 
> d2992c05b221ea90dae1c06d27753932f7411925 
>   3rdparty/stout/include/stout/os/windows/fcntl.hpp 
> bf8c38acad60f9b0eb752053dcd53a9fda7b8bfa 
>   3rdparty/stout/include/stout/os/windows/mktemp.hpp 
> 5c775c45c415d9ddd6a80ab814fb55475e9f871e 
>   3rdparty/stout/include/stout/os/windows/open.hpp PRE-CREATION 
>   3rdparty/stout/include/stout/windows.hpp 
> 1bfcdf4a5c097cc6d2293396ce39c8ad2c9ec993 
> 
> 
> Diff: https://reviews.apache.org/r/66424/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Andrew Schwartzmeyer
> 
>

Re: Review Request 66424: Windows: Replaced `_wopen()` with `CreateFileW()` in `os::open()`.

2018-04-04 Thread John Kordich via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66424/#review200505
---


Ship it!




Ship It!

- John Kordich


On April 4, 2018, 5:47 a.m., Andrew Schwartzmeyer wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66424/
> ---
> 
> (Updated April 4, 2018, 5:47 a.m.)
> 
> 
> Review request for mesos, Akash Gupta, Eric Mumau, John Kordich, Joseph Wu, 
> and Michael Park.
> 
> 
> Bugs: MESOS-8673
> https://issues.apache.org/jira/browse/MESOS-8673
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Instead of using the CRT implementation of `_wopen()` for the
> `os::open()` API, we now use the Windows API `CreateFileW()`, mapping
> each of the Linux `open()` flags to their semantic equivalents. This
> will make implementing overlapped I/O possible, and is a step toward
> removing the use of integer file descriptors on Windows.
> 
> Note that instead of redefining the C flags like `O_RDONLY`, we just
> use them directly in our mapping logic, and set the used but
> unsupported flags to zero.
> 
> This change uncovered several bugs such as incorrect access flags, and
> used-but-not-included headers.
> 
> We currently ignore creation permissions as they will be handled in a
> broader project to map permissions to Windows correctly.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/net.hpp 
> d2992c05b221ea90dae1c06d27753932f7411925 
>   3rdparty/stout/include/stout/os/windows/fcntl.hpp 
> bf8c38acad60f9b0eb752053dcd53a9fda7b8bfa 
>   3rdparty/stout/include/stout/os/windows/mktemp.hpp 
> 5c775c45c415d9ddd6a80ab814fb55475e9f871e 
>   3rdparty/stout/include/stout/os/windows/open.hpp PRE-CREATION 
>   3rdparty/stout/include/stout/windows.hpp 
> 1bfcdf4a5c097cc6d2293396ce39c8ad2c9ec993 
> 
> 
> Diff: https://reviews.apache.org/r/66424/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Andrew Schwartzmeyer
> 
>

Review Request 66424: Windows: Replaced `_wopen()` with `CreateFileW()` in `os::open()`.

2018-04-03 Thread Andrew Schwartzmeyer 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66424/
---

Review request for mesos, Akash Gupta, Eric Mumau, John Kordich, Joseph Wu, and 
Michael Park.


Bugs: MESOS-8673
https://issues.apache.org/jira/browse/MESOS-8673


Repository: mesos


Description
---

Instead of using the CRT implementation of `_wopen()` for the
`os::open()` API, we now use the Windows API `CreateFileW()`, mapping
each of the Linux `open()` flags to their semantic equivalents. This
will make implementing overlapped I/O possible, and is a step toward
removing the use of integer file descriptors on Windows.

Note that instead of redefining the C flags like `O_RDONLY`, we just
use them directly in our mapping logic, and set the used but
unsupported flags to zero.

This change uncovered several bugs such as incorrect access flags, and
used-but-not-included headers.

We currently ignore creation permissions as they will be handled in a
broader project to map permissions to Windows correctly.


Diffs
-

  3rdparty/stout/include/stout/net.hpp d2992c05b221ea90dae1c06d27753932f7411925 
  3rdparty/stout/include/stout/os/windows/fcntl.hpp 
bf8c38acad60f9b0eb752053dcd53a9fda7b8bfa 
  3rdparty/stout/include/stout/os/windows/mktemp.hpp 
5c775c45c415d9ddd6a80ab814fb55475e9f871e 
  3rdparty/stout/include/stout/os/windows/open.hpp PRE-CREATION 
  3rdparty/stout/include/stout/windows.hpp 
1bfcdf4a5c097cc6d2293396ce39c8ad2c9ec993 


Diff: https://reviews.apache.org/r/66424/diff/1/


Testing
---


Thanks,

Andrew Schwartzmeyer