Re: Review Request 64461: Windows: Added legacy support for admin-only symlinks.

2017-12-14 Thread Mesos Reviewbot Windows 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64461/#review193847
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64461']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64461

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64461/logs/mesos-tests-stdout.log):

```

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2287 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2308 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2247 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2270 ms total)

[--] Global test environment tear-down
[==] 835 tests from 85 test cases ran. (297796 ms total)
[  PASSED  ] 825 tests.
[  FAILED  ] 10 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateLatestWhenResending

10 FAILED TESTS
  YOU HAVE 205 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64461/logs/mesos-tests-stderr.log):

```
I1214 21:37:20.438609  8716 master.cpp:10160] Updating the state of task 
365ae104-4811-4fbf-bd45-cd534e7a4389 of framework 
b4c37894-808c-423f-bd55-8db00978f5af- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I1214 21:37:20.439608  3772 slave.cpp:3401] Shutting down framework 
b4c37894-808c-423f-bd55-8db00978f5af-
I1214 21:37:20.439608  3772 slave.cpp:6109] Shutting down executor 
'365ae104-4811-4fbf-bd45-cd534e7a4389' of framework 
b4c37894-808c-423f-bd55-8db00978f5af- at executor(1)@10.3.1.5:49642
I1214 21:37:20.440609  3772 slave.cpp:909] Agent terminating
W1214 21:37:20.440609  3772 slave.cpp:3397] Ignoring shutdown framework 
b4c37894-808c-423f-bd55-8db00978f5af- because it is terminating
I1214 21:37:20.441609  8716 master.cpp:10266] Removing task 
365ae104-4811-4fbf-bd45-cd534e7a4389 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disI1214 21:37:19.795600  7384 exec.cpp:162] Version: 
1.5.0
I1214 21:37:19.820578  8636 exec.cpp:237] Executor registered on agent 
b4c37894-808c-423f-bd55-8db00978f5af-S0
I1214 21:37:19.823597  7060 executor.cpp:171] Received SUBSCRIBED event
I1214 21:37:19.827601  7060 executor.cpp:175] Subscribed executor on 
build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1214 21:37:19.828603  7060 executor.cpp:171] Received LAUNCH event
I1214 21:37:19.831603  7060 executor.cpp:638] Starting task 
365ae104-4811-4fbf-bd45-cd534e7a4389
I1214 21:37:19.905603  7060 executor.cpp:478] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1214 21:37:20.414607  7060 executor.cpp:651] Forked command at 7640
I1214 21:37:20.440609   524 exec.cpp:435] Executor asked to shutdown
I1214 21:37:20.441609  7060 executor.cpp:171] Received SHUTDOWN event
I1214 21:37:20.441609  7060 executor.cpp:748] Shutting down
I1214 21:37:20.441609  7060 executor.cpp:855] Sending SIGTERM to process tree 
at pid 7k(allocated: *):1024; ports(allocated: *):[31000-32000] of framework 
b4c37894-808c-423f-bd55-8db00978f5af- on agent 
b4c37894-808c-423f-bd55-8db00978f5af-S0 at slave(327)@10.3.1.5:49621 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1214 21:37:20.442610  3264 containerizer.cpp:2337] Destroying container 
b9045ef0-ef7f-4db7-9362-91c0f8b2e0e7 in RUNNING state
I1214 21:37:20.443609  3264 containerizer.cpp:2939] Transitioning the state of 
container b9045ef0-ef7f-4db7-9362-91c0f8b2e0e7 from RUNNING to DESTROYING
I1214 21:37:20.443609  8716 master.cpp:1305] Agent 
b4c37894-808c-423f-bd55-8db00978f5af-S0 at slave(327)@10.3.1.5:49621 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1214 21:37:20.443609  8716 master.cpp:3364] Disconnecting agent 
b4c37894-808c-4

Re: Review Request 64461: Windows: Added legacy support for admin-only symlinks.

2017-12-14 Thread James Peach 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64461/#review193845
---


Ship it!




Ship It!

- James Peach


On Dec. 14, 2017, 8:43 p.m., Andrew Schwartzmeyer wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64461/
> ---
> 
> (Updated Dec. 14, 2017, 8:43 p.m.)
> 
> 
> Review request for mesos, Akash Gupta, Alexander Rukletsov, Greg Mann, and 
> Joseph Wu.
> 
> 
> Bugs: MESOS-8307
> https://issues.apache.org/jira/browse/MESOS-8307
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Some users are regrettably stuck on versions of Windows before this
> feature was added. If the `ALLOW_UNPRIVILEGED` flag is unsupported, the
> creation of the symlink will fail with `ERROR_INVALID_PARAMETER`.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/internal/windows/reparsepoint.hpp 
> 158f44550327c63fb416052ad9978b55e4efe36b 
> 
> 
> Diff: https://reviews.apache.org/r/64461/diff/3/
> 
> 
> Testing
> ---
> 
> ~Still works on Windows 10 Fall Creator's Update, needs to be tested to prove 
> working on legacy systems.~
> 
> Tested on Windows Server 2016. _Other, unrelated stout-tests fail on the old 
> systems because of different reasons regarding username expansion._ Symlinks 
> work again.
> 
> 
> Thanks,
> 
> Andrew Schwartzmeyer
> 
>

Re: Review Request 64461: Windows: Added legacy support for admin-only symlinks.

2017-12-14 Thread Andrew Schwartzmeyer 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64461/
---

(Updated Dec. 14, 2017, 12:43 p.m.)


Review request for mesos, Akash Gupta, Alexander Rukletsov, Greg Mann, and 
Joseph Wu.


Changes
---

Style.


Bugs: MESOS-8307
https://issues.apache.org/jira/browse/MESOS-8307


Repository: mesos


Description (updated)
---

Some users are regrettably stuck on versions of Windows before this
feature was added. If the `ALLOW_UNPRIVILEGED` flag is unsupported, the
creation of the symlink will fail with `ERROR_INVALID_PARAMETER`.


Diffs (updated)
-

  3rdparty/stout/include/stout/internal/windows/reparsepoint.hpp 
158f44550327c63fb416052ad9978b55e4efe36b 


Diff: https://reviews.apache.org/r/64461/diff/3/

Changes: https://reviews.apache.org/r/64461/diff/2-3/


Testing
---

~Still works on Windows 10 Fall Creator's Update, needs to be tested to prove 
working on legacy systems.~

Tested on Windows Server 2016. _Other, unrelated stout-tests fail on the old 
systems because of different reasons regarding username expansion._ Symlinks 
work again.


Thanks,

Andrew Schwartzmeyer

Re: Review Request 64461: Windows: Added legacy support for admin-only symlinks.

2017-12-14 Thread Andrew Schwartzmeyer 


> On Dec. 14, 2017, 11:46 a.m., James Peach wrote:
> > 3rdparty/stout/include/stout/internal/windows/reparsepoint.hpp
> > Lines 361 (patched)
> > 
> >
> > Maybe, this control flow would be clearer:
> > 
> > ```C
> > if (link(...)) {
> >return Nothing();
> > }
> > 
> > if (GetLastError() == INVALID) {
> >   if (link(flags))) {
> >   return Nothing();
> >   }
> > }
> > 
> > return WindowsError();
> > ```

That is so much better, thank you.


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64461/#review193832
---


On Dec. 14, 2017, 11:35 a.m., Andrew Schwartzmeyer wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64461/
> ---
> 
> (Updated Dec. 14, 2017, 11:35 a.m.)
> 
> 
> Review request for mesos, Akash Gupta, Alexander Rukletsov, Greg Mann, and 
> Joseph Wu.
> 
> 
> Bugs: MESOS-8307
> https://issues.apache.org/jira/browse/MESOS-8307
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Some users are regrettably stuck on versions of Windows before this
> feature was added. If the `ALLOW_UNPRIVILEGED` flag is unsupported, the
> creation of the symlink will fail with `ERROR_INVALID_PARAMETER`. In
> this case, we try again without the flag (which will require
> administrative privileges, and so need to be guarded by a mutex) for
> these older systems.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/internal/windows/reparsepoint.hpp 
> 158f44550327c63fb416052ad9978b55e4efe36b 
> 
> 
> Diff: https://reviews.apache.org/r/64461/diff/2/
> 
> 
> Testing
> ---
> 
> ~Still works on Windows 10 Fall Creator's Update, needs to be tested to prove 
> working on legacy systems.~
> 
> Tested on Windows Server 2016. _Other, unrelated stout-tests fail on the old 
> systems because of different reasons regarding username expansion._ Symlinks 
> work again.
> 
> 
> Thanks,
> 
> Andrew Schwartzmeyer
> 
>

Re: Review Request 64461: Windows: Added legacy support for admin-only symlinks.

2017-12-14 Thread Mesos Reviewbot Windows 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64461/#review193840
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64461']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64461

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64461/logs/mesos-tests-stdout.log):

```

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2282 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2304 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2265 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2288 ms total)

[--] Global test environment tear-down
[==] 835 tests from 85 test cases ran. (299094 ms total)
[  PASSED  ] 825 tests.
[  FAILED  ] 10 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateLatestWhenResending

10 FAILED TESTS
  YOU HAVE 205 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64461/logs/mesos-tests-stderr.log):

```
I1214 20:37:13.857149  2332 master.cpp:10160] Updating the state of task 
2243b318-a717-40ea-bb08-02a3b51f7f60 of framework 
7efbd17b-922b-4ea0-9ee0-a70d65544f0c- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I1214 20:37:13.857149  4128 slave.cpp:3401] Shutting down framework 
7efbd17b-922b-4ea0-9ee0-a70d65544f0c-
I1214 20:37:13.857149  4128 slave.cpp:6109] Shutting down executor 
'2243b318-a717-40ea-bb08-02a3b51f7f60' of framework 
7efbd17b-922b-4ea0-9ee0-a70d65544f0c- at executor(1)@10.3.1.5:64086
I1214 20:37:13.857149  4128 slave.cpp:909] AgeI1214 20:37:13.193153  1112 
exec.cpp:162] Version: 1.5.0
I1214 20:37:13.216130  8872 exec.cpp:237] Executor registered on agent 
7efbd17b-922b-4ea0-9ee0-a70d65544f0c-S0
I1214 20:37:13.219130  4708 executor.cpp:171] Received SUBSCRIBED event
I1214 20:37:13.223132  4708 executor.cpp:175] Subscribed executor on 
build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1214 20:37:13.224133  4708 executor.cpp:171] Received LAUNCH event
I1214 20:37:13.227135  4708 executor.cpp:638] Starting task 
2243b318-a717-40ea-bb08-02a3b51f7f60
I1214 20:37:13.302156  4708 executor.cpp:478] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1214 20:37:13.833148  4708 executor.cpp:651] Forked command at 5388
I1214 20:37:13.858150  7868 exec.cpp:435] Executor asked to shutdown
I1214 20:37:13.859150  6836 executor.cpp:171] Received SHUTDOWN event
I1214 20:37:13.859150  6836 executor.cpp:748] Shutting down
I1214 20:37:13.859150  6836 executor.cpp:855] Sending SIGTERM to process tree 
at pid 5nt terminating
W1214 20:37:13.858150  4128 slave.cpp:3397] Ignoring shutdown framework 
7efbd17b-922b-4ea0-9ee0-a70d65544f0c- because it is terminating
I1214 20:37:13.859150  2332 master.cpp:10266] Removing task 
2243b318-a717-40ea-bb08-02a3b51f7f60 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework 7efbd17b-922b-4ea0-9ee0-a70d65544f0c- on 
agent 7efbd17b-922b-4ea0-9ee0-a70d65544f0c-S0 at slave(327)@10.3.1.5:64065 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1214 20:37:13.861150  5328 containerizer.cpp:2337] Destroying container 
b1de4686-9058-4e77-825e-6497b8739251 in RUNNING state
I1214 20:37:13.861150  2332 master.cpp:1305] Agent 
7efbd17b-922b-4ea0-9ee0-a70d65544f0c-S0 at slave(327)@10.3.1.5:64065 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1214 20:37:13.861150  5328 containerizer.cpp:2939] Transitioning the state of 
container b1de4686-9058-4e77-825e-6497b8739251 from RUNNING to DESTROYING
I1214 20:37:13.861150  2332 master.cpp:3364] Disconnecting agent 
7efbd17b-922b-4

Re: Review Request 64461: Windows: Added legacy support for admin-only symlinks.

2017-12-14 Thread James Peach 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64461/#review193832
---




3rdparty/stout/include/stout/internal/windows/reparsepoint.hpp
Lines 361 (patched)


Maybe, this control flow would be clearer:

```C
if (link(...)) {
   return Nothing();
}

if (GetLastError() == INVALID) {
  if (link(flags))) {
  return Nothing();
  }
}

return WindowsError();
```


- James Peach


On Dec. 14, 2017, 7:35 p.m., Andrew Schwartzmeyer wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64461/
> ---
> 
> (Updated Dec. 14, 2017, 7:35 p.m.)
> 
> 
> Review request for mesos, Akash Gupta, Alexander Rukletsov, Greg Mann, and 
> Joseph Wu.
> 
> 
> Bugs: MESOS-8307
> https://issues.apache.org/jira/browse/MESOS-8307
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Some users are regrettably stuck on versions of Windows before this
> feature was added. If the `ALLOW_UNPRIVILEGED` flag is unsupported, the
> creation of the symlink will fail with `ERROR_INVALID_PARAMETER`. In
> this case, we try again without the flag (which will require
> administrative privileges, and so need to be guarded by a mutex) for
> these older systems.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/internal/windows/reparsepoint.hpp 
> 158f44550327c63fb416052ad9978b55e4efe36b 
> 
> 
> Diff: https://reviews.apache.org/r/64461/diff/2/
> 
> 
> Testing
> ---
> 
> ~Still works on Windows 10 Fall Creator's Update, needs to be tested to prove 
> working on legacy systems.~
> 
> Tested on Windows Server 2016. _Other, unrelated stout-tests fail on the old 
> systems because of different reasons regarding username expansion._ Symlinks 
> work again.
> 
> 
> Thanks,
> 
> Andrew Schwartzmeyer
> 
>

Re: Review Request 64461: Windows: Added legacy support for admin-only symlinks.

2017-12-14 Thread Andrew Schwartzmeyer 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64461/
---

(Updated Dec. 14, 2017, 11:35 a.m.)


Review request for mesos, Akash Gupta, Alexander Rukletsov, Greg Mann, and 
Joseph Wu.


Bugs: MESOS-8307
https://issues.apache.org/jira/browse/MESOS-8307


Repository: mesos


Description
---

Some users are regrettably stuck on versions of Windows before this
feature was added. If the `ALLOW_UNPRIVILEGED` flag is unsupported, the
creation of the symlink will fail with `ERROR_INVALID_PARAMETER`. In
this case, we try again without the flag (which will require
administrative privileges, and so need to be guarded by a mutex) for
these older systems.


Diffs
-

  3rdparty/stout/include/stout/internal/windows/reparsepoint.hpp 
158f44550327c63fb416052ad9978b55e4efe36b 


Diff: https://reviews.apache.org/r/64461/diff/2/


Testing (updated)
---

~Still works on Windows 10 Fall Creator's Update, needs to be tested to prove 
working on legacy systems.~

Tested on Windows Server 2016. _Other, unrelated stout-tests fail on the old 
systems because of different reasons regarding username expansion._ Symlinks 
work again.


Thanks,

Andrew Schwartzmeyer

Re: Review Request 64461: Windows: Added legacy support for admin-only symlinks.

2017-12-14 Thread Andrew Schwartzmeyer 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64461/
---

(Updated Dec. 14, 2017, 11:23 a.m.)


Review request for mesos, Akash Gupta, Alexander Rukletsov, Greg Mann, and 
Joseph Wu.


Bugs: MESOS-8307
https://issues.apache.org/jira/browse/MESOS-8307


Repository: mesos


Description
---

Some users are regrettably stuck on versions of Windows before this
feature was added. If the `ALLOW_UNPRIVILEGED` flag is unsupported, the
creation of the symlink will fail with `ERROR_INVALID_PARAMETER`. In
this case, we try again without the flag (which will require
administrative privileges, and so need to be guarded by a mutex) for
these older systems.


Diffs (updated)
-

  3rdparty/stout/include/stout/internal/windows/reparsepoint.hpp 
158f44550327c63fb416052ad9978b55e4efe36b 


Diff: https://reviews.apache.org/r/64461/diff/2/

Changes: https://reviews.apache.org/r/64461/diff/1-2/


Testing
---

Still works on Windows 10 Fall Creator's Update, needs to be tested to prove 
working on legacy systems.


Thanks,

Andrew Schwartzmeyer

Re: Review Request 64461: Windows: Added legacy support for admin-only symlinks.

2017-12-12 Thread Andrew Schwartzmeyer 


> On Dec. 8, 2017, 5:47 p.m., Akash Gupta wrote:
> > 3rdparty/stout/include/stout/internal/windows/reparsepoint.hpp
> > Lines 364 (patched)
> > 
> >
> > Could you point out where in the docs that say this mutex is required. 
> > I couldn't find it.
> 
> Andrew Schwartzmeyer wrote:
> I don't honestly know. This came from resurrected code. It's about time 
> to double check its correctness.

As far as I've been able to find, this does not appear to be true.

You either have, as a user, the `SeCreateSymbolicLinkPrivilege` privilege, or 
you don't. If you don't, the call fails with permission denied. If you do, the 
call succeeds. Nothing in the current documentation says that it modifies the 
process's permissions in order to create the link. It certainly doesn't appear 
to prompt like "you need to be an admin to do this, continue?" and elevate 
then. It _seems_ to be all or nothing.

Nothing in the linked `CreateSymlink` documentation mentions this either. I'm 
getting tempted to remove it entirely.


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64461/#review193302
---


On Dec. 8, 2017, 1:18 p.m., Andrew Schwartzmeyer wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64461/
> ---
> 
> (Updated Dec. 8, 2017, 1:18 p.m.)
> 
> 
> Review request for mesos, Akash Gupta, Alexander Rukletsov, Greg Mann, and 
> Joseph Wu.
> 
> 
> Bugs: MESOS-8307
> https://issues.apache.org/jira/browse/MESOS-8307
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Some users are regrettably stuck on versions of Windows before this
> feature was added. If the `ALLOW_UNPRIVILEGED` flag is unsupported, the
> creation of the symlink will fail with `ERROR_INVALID_PARAMETER`. In
> this case, we try again without the flag (which will require
> administrative privileges, and so need to be guarded by a mutex) for
> these older systems.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/internal/windows/reparsepoint.hpp 
> 158f44550327c63fb416052ad9978b55e4efe36b 
> 
> 
> Diff: https://reviews.apache.org/r/64461/diff/1/
> 
> 
> Testing
> ---
> 
> Still works on Windows 10 Fall Creator's Update, needs to be tested to prove 
> working on legacy systems.
> 
> 
> Thanks,
> 
> Andrew Schwartzmeyer
> 
>

Re: Review Request 64461: Windows: Added legacy support for admin-only symlinks.

2017-12-11 Thread Andrew Schwartzmeyer 


> On Dec. 8, 2017, 5:47 p.m., Akash Gupta wrote:
> > 3rdparty/stout/include/stout/internal/windows/reparsepoint.hpp
> > Lines 354 (patched)
> > 
> >
> > Does this flag exist on older versions of Windows? You might need a 
> > `#ifndef X #define X Y #endif` to get this to compile
> 
> Andrew Schwartzmeyer wrote:
> The flag is in the SDK, so it builds correctly as long as Visual Studio 
> is up-to-date. It just fails at runtime on older systems.

Confirmed. If the Visual Studio Windows 10 SDK >= 15063, this will build 
regardless of the Windows version. With it built, this will work at runtime on 
the older systems (pre build 15063).

Example: CloudBase's CI (the ReviewBot) was building fine, but with a 
`mesos-tests` unit test failing at runtime. The symlink stout-tests were 
appropriately skipped because `fs::symlink` failed, but that's not great 
because `mesos-tests` require them anyway, and don't filter. Mesosphere's CI 
was failing at compile-time, but that should be resolvable by updating the SDK.


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64461/#review193302
---


On Dec. 8, 2017, 1:18 p.m., Andrew Schwartzmeyer wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64461/
> ---
> 
> (Updated Dec. 8, 2017, 1:18 p.m.)
> 
> 
> Review request for mesos, Akash Gupta, Alexander Rukletsov, Greg Mann, and 
> Joseph Wu.
> 
> 
> Bugs: MESOS-8307
> https://issues.apache.org/jira/browse/MESOS-8307
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Some users are regrettably stuck on versions of Windows before this
> feature was added. If the `ALLOW_UNPRIVILEGED` flag is unsupported, the
> creation of the symlink will fail with `ERROR_INVALID_PARAMETER`. In
> this case, we try again without the flag (which will require
> administrative privileges, and so need to be guarded by a mutex) for
> these older systems.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/internal/windows/reparsepoint.hpp 
> 158f44550327c63fb416052ad9978b55e4efe36b 
> 
> 
> Diff: https://reviews.apache.org/r/64461/diff/1/
> 
> 
> Testing
> ---
> 
> Still works on Windows 10 Fall Creator's Update, needs to be tested to prove 
> working on legacy systems.
> 
> 
> Thanks,
> 
> Andrew Schwartzmeyer
> 
>

Re: Review Request 64461: Windows: Added legacy support for admin-only symlinks.

2017-12-11 Thread Mesos Reviewbot Windows 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64461/#review193414
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64461']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64461

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64461/logs/mesos-tests-stdout.log):

```

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2372 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2394 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2469 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2494 ms total)

[--] Global test environment tear-down
[==] 825 tests from 84 test cases ran. (321166 ms total)
[  PASSED  ] 815 tests.
[  FAILED  ] 10 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] SlaveTest.ResourceProviderPublishAll

10 FAILED TESTS
  YOU HAVE 201 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64461/logs/mesos-tests-stderr.log):

```
I1211 18:06:13.566877  3136 exec.cpp:237] Executor registered on agent 
2fa97121-5623-4dcb-90a3-eddfc7d105ab-S0
I1211 18:06:13.569878  8296 executor.cpp:171] Received SUBSCRIBED event
I1211 18:06:13.573878  8296 executor.cpp:175] Subscribed executor on 
build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1211 18:06:13.573878  8296 executor.cpp:171] Received LAUNCH event
I1211 18:06:13.577877  8296 executor.cpp:637] Starting task 
f740d1f5-f30d-4b36-8195-59a73d07110d
I1211 18:06:13.656875  8296 executor.cpp:477] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1211 18:06:14.214869  8296 executor.cpp:650] Forked command at 5336
I1211 18:06:14.247869  5840 exec.cpp:435] Executor asked to shutdown
I1211 18:06:14.248868  8296 executor.cpp:171] Received SHUTDOWN event
I1211 18:06:14.248868  8296 executor.cpp:747] Shutting down
I1211 18:06:14.248868  8296 executor.cpp:854] Sending SIGTERM to process tree 
at pid 5- (latest state: TASK_KILLED, status update state: TASK_KILLED)
I1211 18:06:14.240901 10132 hierarchical.cpp:405] Deactivated framework 
2fa97121-5623-4dcb-90a3-eddfc7d105ab-
I1211 18:06:14.243893  5968 slave.cpp:3400] Shutting down framework 
2fa97121-5623-4dcb-90a3-eddfc7d105ab-
I1211 18:06:14.246897  5968 slave.cpp:6091] Shutting down executor 
'f740d1f5-f30d-4b36-8195-59a73d07110d' of framework 
2fa97121-5623-4dcb-90a3-eddfc7d105ab- at executor(1)@10.3.1.11:52191
I1211 18:06:14.247869  5968 slave.cpp:909] Agent terminating
W1211 18:06:14.247869  5968 slave.cpp:3396] Ignoring shutdown framework 
2fa97121-5623-4dcb-90a3-eddfc7d105ab- because it is terminating
I1211 18:06:14.248868  4728 master.cpp:10209] Removing task 
f740d1f5-f30d-4b36-8195-59a73d07110d with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework 2fa97121-5623-4dcb-90a3-eddfc7d105ab- on 
agent 2fa97121-5623-4dcb-90a3-eddfc7d105ab-S0 at slave(326)@10.3.1.11:52170 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1211 18:06:14.250895  8000 containerizer.cpp:2328] Destroying container 
12a08478-55cf-4037-a87d-4ed9628738e5 in RUNNING state
I1211 18:06:14.250895  8000 containerizer.cpp:2930] Transitioning the state of 
container 12a08478-55cf-4037-a87d-4ed9628738e5 from RUNNING to DESTROYING
I1211 18:06:14.251873  4728 master.cpp:1310] Agent 
2fa97121-5623-4dcb-90a3-eddfc7d105ab-S0 at slave(326)@10.3.1.11:52170 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1211 18:06:14.251873  4728 master.cpp:3369] Disconnecting agent 
2fa97121-5623-4dcb-90a3-eddfc7d105ab-S0 at slave(326)@10.3.1.11:52170 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1211

Re: Review Request 64461: Windows: Added legacy support for admin-only symlinks.

2017-12-08 Thread Andrew Schwartzmeyer 


> On Dec. 8, 2017, 5:47 p.m., Akash Gupta wrote:
> > 3rdparty/stout/include/stout/internal/windows/reparsepoint.hpp
> > Lines 354 (patched)
> > 
> >
> > Does this flag exist on older versions of Windows? You might need a 
> > `#ifndef X #define X Y #endif` to get this to compile

The flag is in the SDK, so it builds correctly as long as Visual Studio is 
up-to-date. It just fails at runtime on older systems.


> On Dec. 8, 2017, 5:47 p.m., Akash Gupta wrote:
> > 3rdparty/stout/include/stout/internal/windows/reparsepoint.hpp
> > Lines 364 (patched)
> > 
> >
> > Could you point out where in the docs that say this mutex is required. 
> > I couldn't find it.

I don't honestly know. This came from resurrected code. It's about time to 
double check its correctness.


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64461/#review193302
---


On Dec. 8, 2017, 1:18 p.m., Andrew Schwartzmeyer wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64461/
> ---
> 
> (Updated Dec. 8, 2017, 1:18 p.m.)
> 
> 
> Review request for mesos, Akash Gupta, Alexander Rukletsov, Greg Mann, and 
> Joseph Wu.
> 
> 
> Bugs: MESOS-8307
> https://issues.apache.org/jira/browse/MESOS-8307
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Some users are regrettably stuck on versions of Windows before this
> feature was added. If the `ALLOW_UNPRIVILEGED` flag is unsupported, the
> creation of the symlink will fail with `ERROR_INVALID_PARAMETER`. In
> this case, we try again without the flag (which will require
> administrative privileges, and so need to be guarded by a mutex) for
> these older systems.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/internal/windows/reparsepoint.hpp 
> 158f44550327c63fb416052ad9978b55e4efe36b 
> 
> 
> Diff: https://reviews.apache.org/r/64461/diff/1/
> 
> 
> Testing
> ---
> 
> Still works on Windows 10 Fall Creator's Update, needs to be tested to prove 
> working on legacy systems.
> 
> 
> Thanks,
> 
> Andrew Schwartzmeyer
> 
>

Re: Review Request 64461: Windows: Added legacy support for admin-only symlinks.

2017-12-08 Thread Akash Gupta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64461/#review193302
---



Also, we need to update the tests or fix the library functions because 
sometimes we get short paths like `ADMIN~1` vs `Administrator` depending on 
what Windows API we call. Causing stuff like this:
[  FAILED  ] FsTest.CreateDirectoryAtMaxPath
[  FAILED  ] FsTest.CreateDirectoryLongerThanMaxPath


3rdparty/stout/include/stout/internal/windows/reparsepoint.hpp
Lines 354 (patched)


Does this flag exist on older versions of Windows? You might need a 
`#ifndef X #define X Y #endif` to get this to compile



3rdparty/stout/include/stout/internal/windows/reparsepoint.hpp
Lines 364 (patched)


Could you point out where in the docs that say this mutex is required. I 
couldn't find it.


- Akash Gupta


On Dec. 8, 2017, 9:18 p.m., Andrew Schwartzmeyer wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64461/
> ---
> 
> (Updated Dec. 8, 2017, 9:18 p.m.)
> 
> 
> Review request for mesos, Akash Gupta, Alexander Rukletsov, Greg Mann, and 
> Joseph Wu.
> 
> 
> Bugs: MESOS-8307
> https://issues.apache.org/jira/browse/MESOS-8307
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Some users are regrettably stuck on versions of Windows before this
> feature was added. If the `ALLOW_UNPRIVILEGED` flag is unsupported, the
> creation of the symlink will fail with `ERROR_INVALID_PARAMETER`. In
> this case, we try again without the flag (which will require
> administrative privileges, and so need to be guarded by a mutex) for
> these older systems.
> 
> 
> Diffs
> -
> 
>   3rdparty/stout/include/stout/internal/windows/reparsepoint.hpp 
> 158f44550327c63fb416052ad9978b55e4efe36b 
> 
> 
> Diff: https://reviews.apache.org/r/64461/diff/1/
> 
> 
> Testing
> ---
> 
> Still works on Windows 10 Fall Creator's Update, needs to be tested to prove 
> working on legacy systems.
> 
> 
> Thanks,
> 
> Andrew Schwartzmeyer
> 
>

Review Request 64461: Windows: Added legacy support for admin-only symlinks.

2017-12-08 Thread Andrew Schwartzmeyer 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64461/
---

Review request for mesos, Akash Gupta, Alexander Rukletsov, Greg Mann, and 
Joseph Wu.


Bugs: MESOS-8307
https://issues.apache.org/jira/browse/MESOS-8307


Repository: mesos


Description
---

Some users are regrettably stuck on versions of Windows before this
feature was added. If the `ALLOW_UNPRIVILEGED` flag is unsupported, the
creation of the symlink will fail with `ERROR_INVALID_PARAMETER`. In
this case, we try again without the flag (which will require
administrative privileges, and so need to be guarded by a mutex) for
these older systems.


Diffs
-

  3rdparty/stout/include/stout/internal/windows/reparsepoint.hpp 
158f44550327c63fb416052ad9978b55e4efe36b 


Diff: https://reviews.apache.org/r/64461/diff/1/


Testing
---

Still works on Windows 10 Fall Creator's Update, needs to be tested to prove 
working on legacy systems.


Thanks,

Andrew Schwartzmeyer