Re: Review Request 64467: Rewrote the quota headroom enforcement logic in the allocator.

2017-12-12 Thread Meng Zhu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64467/
---

(Updated Dec. 12, 2017, 10:54 p.m.)


Review request for mesos, Benjamin Mahler and Michael Park.


Changes
---

Thank you! Patch updated, comments addressed.


Bugs: MESOS-8293
https://issues.apache.org/jira/browse/MESOS-8293


Repository: mesos


Description
---

Now before offering unreserved resources to frameworks, the
resources are holdout for the quota headroom until the headroom
is met (reserved resources are offered unaffected).


Diffs (updated)
-

  src/master/allocator/mesos/hierarchical.cpp 
2b2d1fd2802203eba482be2992a5f2756d100cbf 
  src/tests/hierarchical_allocator_tests.cpp 
862f4683da04d37d9fe9f471d6ec9cd7751f39ec 


Diff: https://reviews.apache.org/r/64467/diff/4/

Changes: https://reviews.apache.org/r/64467/diff/3-4/


Testing
---

make check and a dediated test in #64465


Thanks,

Meng Zhu

Re: Review Request 64558: Added a missing check for parent container has config recovered.

2017-12-12 Thread Gilbert Song 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64558/#review193654
---




src/slave/containerizer/mesos/containerizer.cpp
Line 1723 (original), 1723 (patched)


This check is not needed since the check above is necessary.


- Gilbert Song


On Dec. 12, 2017, 4:33 p.m., Zhitao Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64558/
> ---
> 
> (Updated Dec. 12, 2017, 4:33 p.m.)
> 
> 
> Review request for mesos, Gilbert Song and Greg Mann.
> 
> 
> Bugs: MESOS-8325
> https://issues.apache.org/jira/browse/MESOS-8325
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added a missing check for parent container has config recovered.
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/mesos/containerizer.cpp 
> a67311734a7cd8155ba54fe9ac45b7a0e3312168 
> 
> 
> Diff: https://reviews.apache.org/r/64558/diff/1/
> 
> 
> Testing
> ---
> 
> make
> 
> 
> Thanks,
> 
> Zhitao Li
> 
>

Re: Review Request 64558: Added a missing check for parent container has config recovered.

2017-12-12 Thread Gilbert Song 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64558/#review193652
---


Fix it, then Ship it!





src/slave/containerizer/mesos/containerizer.cpp
Line 1716 (original), 1716 (patched)


Need to guard this `check`, right?


- Gilbert Song


On Dec. 12, 2017, 4:33 p.m., Zhitao Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64558/
> ---
> 
> (Updated Dec. 12, 2017, 4:33 p.m.)
> 
> 
> Review request for mesos, Gilbert Song and Greg Mann.
> 
> 
> Bugs: MESOS-8325
> https://issues.apache.org/jira/browse/MESOS-8325
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added a missing check for parent container has config recovered.
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/mesos/containerizer.cpp 
> a67311734a7cd8155ba54fe9ac45b7a0e3312168 
> 
> 
> Diff: https://reviews.apache.org/r/64558/diff/1/
> 
> 
> Testing
> ---
> 
> make
> 
> 
> Thanks,
> 
> Zhitao Li
> 
>

Re: Review Request 64551: Integrated the status update manager into SLRP.

2017-12-12 Thread Mesos Reviewbot Windows 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64551/#review193645
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64521', '64475', '64559', '64551']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64551

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64551/logs/mesos-tests-stdout.log):

```
[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2325 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2346 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2268 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2290 ms total)

[--] Global test environment tear-down
[==] 830 tests from 84 test cases ran. (304861 ms total)
[  PASSED  ] 819 tests.
[  FAILED  ] 11 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateLatestWhenResending
[  FAILED  ] SlaveTest.ResourceProviderPublishAll

11 FAILED TESTS
  YOU HAVE 204 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64551/logs/mesos-tests-stderr.log):

```
I1213 05:00:35.168634  4056 slave.cpp:3400] Shutting down framework 
10eaf617-8e77-4711-92ae-f97267c755dc-
I1213 05:00:35.168634  4948 master.cpp:10158] Updating the state of task 
e998c345-6629-43c0-bad6-7d5b81830efe of framework 
10eaf617-8e77-4711-92ae-f97267c755dc- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I1213 05:00:35.168634  4056 slave.cpp:6114] ShuttingI1213 05:00:34.505656  2112 
exec.cpp:162] Version: 1.5.0
I1213 05:00:34.529639  3796 exec.cpp:237] Executor registered on agent 
10eaf617-8e77-4711-92ae-f97267c755dc-S0
I1213 05:00:34.532636  5168 executor.cpp:171] Received SUBSCRIBED event
I1213 05:00:34.536660  5168 executor.cpp:175] Subscribed executor on 
build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1213 05:00:34.536660  5168 executor.cpp:171] Received LAUNCH event
I1213 05:00:34.540634  5168 executor.cpp:637] Starting task 
e998c345-6629-43c0-bad6-7d5b81830efe
I1213 05:00:34.617655  5168 executor.cpp:477] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1213 05:00:35.144623  5168 executor.cpp:650] Forked command at 5304
I1213 05:00:35.170622  7000 exec.cpp:435] Executor asked to shutdown
I1213 05:00:35.17062296 executor.cpp:171] Received SHUTDOWN event
I1213 05:00:35.17162296 executor.cpp:747] Shutting down
I1213 05:00:35.17162296 executor.cpp:854] Sending SIGTERM to process tree 
at pid 5 down executor 'e998c345-6629-43c0-bad6-7d5b81830efe' of framework 
10eaf617-8e77-4711-92ae-f97267c755dc- at executor(1)@10.3.1.5:61661
I1213 05:00:35.169622  4056 slave.cpp:909] Agent terminating
W1213 05:00:35.170622  4056 slave.cpp:3396] Ignoring shutdown framework 
10eaf617-8e77-4711-92ae-f97267c755dc- because it is terminating
I1213 05:00:35.171622  4948 master.cpp:10264] Removing task 
e998c345-6629-43c0-bad6-7d5b81830efe with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework 10eaf617-8e77-4711-92ae-f97267c755dc- on 
agent 10eaf617-8e77-4711-92ae-f97267c755dc-S0 at slave(326)@10.3.1.5:61639 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1213 05:00:35.173622  1100 containerizer.cpp:2328] Destroying container 
4c540913-06d7-481c-b7b8-4db94e2219be in RUNNING state
I1213 05:00:35.173622  1100 containerizer.cpp:2930] Transitioning the state of 
container 4c540913-06d7-481c-b7b8-4db94e2219be from RUNNING to DESTROYING
I1213 05:00:35.173622  4948 master.cpp:1305] Agent 
10eaf617-8e77-4711-92ae-f97267c755dc-S0 at slave(326)@10.3.1.5:61639 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1213

Re: Review Request 64561: Removed resource categories in UpdateSlaveMessage.

2017-12-12 Thread Mesos Reviewbot Windows 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64561/#review193641
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64557', '64561']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64561

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64561/logs/mesos-tests-stdout.log):

```

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2430 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2455 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2373 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2397 ms total)

[--] Global test environment tear-down
[==] 831 tests from 84 test cases ran. (323528 ms total)
[  PASSED  ] 821 tests.
[  FAILED  ] 10 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] SlaveTest.ResourceProviderPublishAll

10 FAILED TESTS
  YOU HAVE 201 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64561/logs/mesos-tests-stderr.log):

```
I1213 04:22:15.114867  3156 master.cpp:10154] Updating the state of task 
ad428d78-1454-409e-972f-2ae2f2f736e9 of framework 
c7b0dfca-1739-4662-8320-06724dd392ec- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I1213 04:22:15.115867  5100 slave.cpp:3400] Shutting down framework 
c7b0dfca-1739-4662-8320-06724dd392ec-
I1213 04:22:15.115867  5100 slave.cpp:6114] SI1213 04:22:14.415889  5280 
exec.cpp:162] Version: 1.5.0
I1213 04:22:14.439934  1604 exec.cpp:237] Executor registered on agent 
c7b0dfca-1739-4662-8320-06724dd392ec-S0
I1213 04:22:14.442921  6824 executor.cpp:171] Received SUBSCRIBED event
I1213 04:22:14.446918  6824 executor.cpp:175] Subscribed executor on 
build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1213 04:22:14.447918  6824 executor.cpp:171] Received LAUNCH event
I1213 04:22:14.451931  6824 executor.cpp:637] Starting task 
ad428d78-1454-409e-972f-2ae2f2f736e9
I1213 04:22:14.533891  6824 executor.cpp:477] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1213 04:22:15.088878  6824 executor.cpp:650] Forked command at 3060
I1213 04:22:15.117868  2428 exec.cpp:435] Executor asked to shutdown
I1213 04:22:15.117868  6824 executor.cpp:171] Received SHUTDOWN event
I1213 04:22:15.117868  6824 executor.cpp:747] Shutting down
I1213 04:22:15.117868  6824 executor.cpp:854] Sending SIGTERM to process tree 
at pid 3hutting down executor 'ad428d78-1454-409e-972f-2ae2f2f736e9' of 
framework c7b0dfca-1739-4662-8320-06724dd392ec- at 
executor(1)@10.3.1.11:61392
I1213 04:22:15.116868  5100 slave.cpp:909] Agent terminating
W1213 04:22:15.116868  5100 slave.cpp:3396] Ignoring shutdown framework 
c7b0dfca-1739-4662-8320-06724dd392ec- because it is terminating
I1213 04:22:15.117868  3156 master.cpp:10260] Removing task 
ad428d78-1454-409e-972f-2ae2f2f736e9 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework c7b0dfca-1739-4662-8320-06724dd392ec- on 
agent c7b0dfca-1739-4662-8320-06724dd392ec-S0 at slave(328)@10.3.1.11:61371 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1213 04:22:15.119868  3156 master.cpp:1305] Agent 
c7b0dfca-1739-4662-8320-06724dd392ec-S0 at slave(328)@10.3.1.11:61371 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1213 04:22:15.119868  3156 master.cpp:3364] Disconnecting agent 
c7b0dfca-1739-4662-8320-06724dd392ec-S0 at slave(328)@10.3.1.11:61371 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1213 04:22:15.119868  9832 containerizer.cpp:2328] Destroying container 
34f12d3c-f5e9-496b-be50-1f57768b3e80 in RUNNING state
I1213 04:22:15.120872  6840

Re: Review Request 64387: Windows: Ported docker health check tests.

2017-12-12 Thread Mesos Reviewbot Windows 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64387/#review193640
---



FAIL: Some Mesos tests failed.

Reviews applied: `['63859', '63860', '63861', '64386', '64387']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64387

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64387/logs/mesos-tests-stdout.log):

```
[ RUN  ] FilesTest.DownloadTest
[   OK ] FilesTest.DownloadTest (52 ms)
[ RUN  ] FilesTest.DebugTest
[   OK ] FilesTest.DebugTest (39 ms)
[ RUN  ] FilesTest.AuthenticationTest
[   OK ] FilesTest.AuthenticationTest (37 ms)
[--] 6 tests from FilesTest (264 ms total)

[--] 3 tests from GarbageCollectorTest
[ RUN  ] GarbageCollectorTest.Schedule
[   OK ] GarbageCollectorTest.Schedule (57 ms)
[ RUN  ] GarbageCollectorTest.Unschedule
[   OK ] GarbageCollectorTest.Unschedule (10 ms)
[ RUN  ] GarbageCollectorTest.Prune
[   OK ] GarbageCollectorTest.Prune (11 ms)
[--] 3 tests from GarbageCollectorTest (81 ms total)

[--] 4 tests from GarbageCollectorIntegrationTest
[ RUN  ] GarbageCollectorIntegrationTest.ExitedFramework
[   OK ] GarbageCollectorIntegrationTest.ExitedFramework (272 ms)
[ RUN  ] GarbageCollectorIntegrationTest.ExitedExecutor
[   OK ] GarbageCollectorIntegrationTest.ExitedExecutor (302 ms)
[ RUN  ] GarbageCollectorIntegrationTest.DiskUsage
[   OK ] GarbageCollectorIntegrationTest.DiskUsage (265 ms)
[ RUN  ] GarbageCollectorIntegrationTest.Unschedule
[   OK ] GarbageCollectorIntegrationTest.Unschedule (293 ms)
[--] 4 tests from GarbageCollectorIntegrationTest (1183 ms total)

[--] 12 tests from HealthCheckTest
```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64387/logs/mesos-tests-stderr.log):

```
I1213 04:12:58.088506  6004 hierarchical.cpp:344] Removed framework 
5998db7e-4992-4f16-83b3-12e9ad361525-
I1213 04:12:58.088506  4652 master.cpp:3383] Deactivating agent 
5998db7e-4992-4f16-83b3-12e9ad361525-S0 at slave(48)@10.3.1.5:59556 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1213 04:12:58.088506  6004 hierarchical.cpp:766] Agent 
5998db7e-4992-4f16-83b3-12e9ad361525-S0 deactivated
I1213 04:12:58.103510  1796 master.cpp:1147] Master terminating
I1213 04:12:58.105509  7144 hierarchical.cpp:609] Removed agent 
5998db7e-4992-4f16-83b3-12e9ad361525-S0
W1213 04:12:58.144517  1796 health_check_tests.cpp:138] Downloading 
akagup/pwsh-nano-admin. This may take a while...
F1213 04:12:58.145517  1796 health_check_tests.cpp:146] CHECK_READY(img): is 
FAILED: Failed to create subprocess 'docker -H npipe:./pip*** Check failure 
stack trace: ***
@   7FF771D558FB  google::LogMessage::Fail
@   7FF771D55800  google::LogMessage::SendToLog
@   7FF771D54FE7  google::LogMessage::Flush
@   7FF771D56B21  google::LogMessageFatal::~LogMessageFatal
@   7FF76DB9BFC7  _CheckFatal::~_CheckFatal
@   7FF76E99E24B  mesos::internal::tests::HealthCheckTest::SetUpTestCase
@   7FF77282E906  testing::TestCase::RunSetUpTestCase
@   7FF77280A711  
testing::internal::HandleSehExceptionsInMethodIfSupported
@   7FF77280A34D  
testing::internal::HandleExceptionsInMethodIfSupported
@   7FF7727EA1F2  testing::TestCase::Run
@   7FF7727F13CB  testing::internal::UnitTestImpl::RunAllTests
@   7FF77280A8B1  
testing::internal::HandleSehExceptionsInMethodIfSupported
@   7FF77280A56D  
testing::internal::HandleExceptionsInMethodIfSupported
@   7FF7727EA935  testing::UnitTest::Run
@   7FF76DED10C2  RUN_ALL_TESTS
@   7FF76DECE5BC  main
@   7FF7728BDC04  invoke_main
@   7FF7728BDAF7  __scrt_common_main_seh
@   7FF7728BD9BE  __scrt_common_main
@   7FF7728BDC99  mainCRTStartup
@   7FFD4B841FE4  BaseThreadInitThunk
@   7FFD4E3CEF91  RtlUserThreadStart
```

- Mesos Reviewbot Windows


On Dec. 12, 2017, 5:48 p.m., Akash Gupta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64387/
> ---
> 
> (Updated Dec. 12, 2017, 5:48 p.m.)
> 
> 
> Review request for mesos, Andrew Schwartzmeyer, Jie Yu, Joseph Wu, and 
> Michael Park.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> The `HealthCheckTest.ROOT_DOCKER_*` and
> `DockerContainerizerHealthCheckTest.*`

Re: Review Request 64559: Checkpointing `OfferOperation` in resource provider states.

2017-12-12 Thread Chun-Hung Hsiao 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64559/
---

(Updated Dec. 13, 2017, 2:59 a.m.)


Review request for mesos and Jie Yu.


Repository: mesos


Description
---

Instead of checkpointing `ApplyOfferOperation`, we now checkpoint
`OfferOperations` in resource provider states such that we can keep
track of completed operations as well.

This patch also does some code cleanup, and modifies a unit test for
storage local resource provider to issue operations in batches.


Diffs (updated)
-

  src/resource_provider/state.proto 321201e0aee9c104e831a6fa91f737216d8cd59a 
  src/resource_provider/storage/provider.cpp 
e806f44ef33405d4a2b133576c60be56e9fe3435 
  src/tests/storage_local_resource_provider_tests.cpp 
e5868bfcb71977d2b2a146865f12a5207010ae40 


Diff: https://reviews.apache.org/r/64559/diff/2/

Changes: https://reviews.apache.org/r/64559/diff/1-2/


Testing
---

sudo make check


Thanks,

Chun-Hung Hsiao

Re: Review Request 64551: Integrated the status update manager into SLRP.

2017-12-12 Thread Chun-Hung Hsiao 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64551/
---

(Updated Dec. 13, 2017, 2:59 a.m.)


Review request for mesos, Gaston Kleiman, Greg Mann, and Jie Yu.


Repository: mesos


Description
---

This patch uses the status update manager to send status updates and
recover operations that are checkpointed as completed in the status
update manager but still in the pending list in SLRP. It also forwards
the acknowledgements to the status update manager, and garbage collect
the metadata for offer operations.


Diffs (updated)
-

  src/resource_provider/daemon.hpp 7c513a2b7c7ca958f9ea57326493f74df5b67c8f 
  src/resource_provider/daemon.cpp 7c783e309672a2d93532fb17f13afcb96d5a7006 
  src/resource_provider/local.hpp 46111d9f8969ea47e695d08832cb5a9ea60d1971 
  src/resource_provider/local.cpp d1d683534d186e77a2beb32c978f2aaea289b086 
  src/resource_provider/storage/provider.hpp 
374f837ad693820afffb86bd1e54915f6796d088 
  src/resource_provider/storage/provider.cpp 
e806f44ef33405d4a2b133576c60be56e9fe3435 


Diff: https://reviews.apache.org/r/64551/diff/2/

Changes: https://reviews.apache.org/r/64551/diff/1-2/


Testing
---

sudo make check


Thanks,

Chun-Hung Hsiao

Re: Review Request 64493: Added tests for quota enforcement with unallocated reservations.

2017-12-12 Thread Meng Zhu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64493/
---

(Updated Dec. 12, 2017, 6:53 p.m.)


Review request for mesos, Benjamin Mahler and Michael Park.


Repository: mesos


Description
---

Added tests for quota enforcement with unallocated reservations.


Diffs (updated)
-

  src/tests/hierarchical_allocator_tests.cpp 
862f4683da04d37d9fe9f471d6ec9cd7751f39ec 


Diff: https://reviews.apache.org/r/64493/diff/3/

Changes: https://reviews.apache.org/r/64493/diff/2-3/


Testing
---

maek check


Thanks,

Meng Zhu

Re: Review Request 64475: Initialized offer operation status update manager in SLRP.

2017-12-12 Thread Gaston Kleiman 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64475/#review193637
---


Ship it!




Ship It!

- Gaston Kleiman


On Dec. 12, 2017, 6:33 p.m., Chun-Hung Hsiao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64475/
> ---
> 
> (Updated Dec. 12, 2017, 6:33 p.m.)
> 
> 
> Review request for mesos, Gaston Kleiman, Greg Mann, and Jie Yu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch adds an agent filesystem layout for checkpointing offer
> operation status updates for resource providers, and initialized
> a status update manager in storage local resource provider.
> 
> 
> Diffs
> -
> 
>   src/resource_provider/storage/provider.cpp 
> e806f44ef33405d4a2b133576c60be56e9fe3435 
>   src/slave/paths.hpp d645d871c36bbe8e766a98650f6aa23b6eab65d8 
>   src/slave/paths.cpp b8004e76964abc210820368a89dbfa6928ef7bfd 
> 
> 
> Diff: https://reviews.apache.org/r/64475/diff/4/
> 
> 
> Testing
> ---
> 
> sudo make check
> 
> 
> Thanks,
> 
> Chun-Hung Hsiao
> 
>

Re: Review Request 64558: Added a missing check for parent container has config recovered.

2017-12-12 Thread Mesos Reviewbot Windows 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64558/#review193638
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64558']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64558

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64558/logs/mesos-tests-stdout.log):

```

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2516 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2539 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2484 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2508 ms total)

[--] Global test environment tear-down
[==] 829 tests from 84 test cases ran. (331078 ms total)
[  PASSED  ] 819 tests.
[  FAILED  ] 10 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] SlaveTest.ResourceProviderPublishAll

10 FAILED TESTS
  YOU HAVE 201 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64558/logs/mesos-tests-stderr.log):

```
I1213 02:34:55.545016  9280 master.cpp:10158] Updating the state of task 
0fa8ae69-6c97-46a9-a870-2cf41f116606 of framework 
7ac45e93-4fc2-4f6a-87a9-a1f6cca2c5fb- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I1213 02:34:55.545016  4464 slave.cpp:3400] Shutting down framework 
7ac45e93-4fc2-4f6a-87a9-a1f6cca2c5fb-
I1213 02:34:55.545994  4464 slave.cpp:6114] Shutting downI1213 02:34:54.812026  
4816 exec.cpp:162] Version: 1.5.0
I1213 02:34:54.838026  6312 exec.cpp:237] Executor registered on agent 
7ac45e93-4fc2-4f6a-87a9-a1f6cca2c5fb-S0
I1213 02:34:54.841059   552 executor.cpp:171] Received SUBSCRIBED event
I1213 02:34:54.845744   552 executor.cpp:175] Subscribed executor on 
build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1213 02:34:54.846029   552 executor.cpp:171] Received LAUNCH event
I1213 02:34:54.850072   552 executor.cpp:637] Starting task 
0fa8ae69-6c97-46a9-a870-2cf41f116606
I1213 02:34:54.946045   552 executor.cpp:477] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1213 02:34:55.518997   552 executor.cpp:650] Forked command at 9448
I1213 02:34:55.547992  1312 exec.cpp:435] Executor asked to shutdown
I1213 02:34:55.547992   552 executor.cpp:171] Received SHUTDOWN event
I1213 02:34:55.547992   552 executor.cpp:747] Shutting down
I1213 02:34:55.547992   552 executor.cpp:854] Sending SIGTERM to process tree 
at pid 9 executor '0fa8ae69-6c97-46a9-a870-2cf41f116606' of framework 
7ac45e93-4fc2-4f6a-87a9-a1f6cca2c5fb- at executor(1)@10.3.1.11:57572
I1213 02:34:55.546994  4464 slave.cpp:909] Agent terminating
W1213 02:34:55.546994  4464 slave.cpp:3396] Ignoring shutdown framework 
7ac45e93-4fc2-4f6a-87a9-a1f6cca2c5fb- because it is terminating
I1213 02:34:55.547992  9280 master.cpp:10264] Removing task 
0fa8ae69-6c97-46a9-a870-2cf41f116606 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework 7ac45e93-4fc2-4f6a-87a9-a1f6cca2c5fb- on 
agent 7ac45e93-4fc2-4f6a-87a9-a1f6cca2c5fb-S0 at slave(326)@10.3.1.11:57551 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1213 02:34:55.549991  7216 containerizer.cpp:2329] Destroying container 
d01c28fc-12a4-45d1-82c1-f25c533bc34e in RUNNING state
I1213 02:34:55.549991  7216 containerizer.cpp:2931] Transitioning the state of 
container d01c28fc-12a4-45d1-82c1-f25c533bc34e from RUNNING to DESTROYING
I1213 02:34:55.549991  9280 master.cpp:1305] Agent 
7ac45e93-4fc2-4f6a-87a9-a1f6cca2c5fb-S0 at slave(326)@10.3.1.11:57551 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1213 02:34:55.550994  9280 master.cpp:3364] Disconnecting agent 
7ac45e93-4fc2-4f6a-87a9-a1f6cca2c5fb-S0

Re: Review Request 64475: Initialized offer operation status update manager in SLRP.

2017-12-12 Thread Chun-Hung Hsiao 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64475/
---

(Updated Dec. 13, 2017, 2:33 a.m.)


Review request for mesos, Gaston Kleiman, Greg Mann, and Jie Yu.


Changes
---

Fixed a typo. Thanks Gaston!


Repository: mesos


Description
---

This patch adds an agent filesystem layout for checkpointing offer
operation status updates for resource providers, and initialized
a status update manager in storage local resource provider.


Diffs (updated)
-

  src/resource_provider/storage/provider.cpp 
e806f44ef33405d4a2b133576c60be56e9fe3435 
  src/slave/paths.hpp d645d871c36bbe8e766a98650f6aa23b6eab65d8 
  src/slave/paths.cpp b8004e76964abc210820368a89dbfa6928ef7bfd 


Diff: https://reviews.apache.org/r/64475/diff/4/

Changes: https://reviews.apache.org/r/64475/diff/3-4/


Testing
---

sudo make check


Thanks,

Chun-Hung Hsiao

Re: Review Request 64559: Checkpointing `OfferOperation` in resource provider states.

2017-12-12 Thread Mesos Reviewbot Windows 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64559/#review193636
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64521', '64475', '64559']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64559

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64559/logs/mesos-tests-stdout.log):

```
[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2394 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2416 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2377 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2399 ms total)

[--] Global test environment tear-down
[==] 830 tests from 84 test cases ran. (308749 ms total)
[  PASSED  ] 819 tests.
[  FAILED  ] 11 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateLatestWhenResending
[  FAILED  ] SlaveTest.ResourceProviderPublishAll

11 FAILED TESTS
  YOU HAVE 204 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64559/logs/mesos-tests-stderr.log):

```
I1213 02:15:25.116502   452 master.cpp:10158] Updating the state of task 
1537bacb-6889-48e5-a95d-02b62f947f6a of framework 
c2c5066d-c610-4aa2-b10f-53065224bc8d- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I1213 02:15:25.116502  8180 slave.cpp:3400] Shutting down framework 
c2c5066d-c610-4aa2-b10f-53065224bc8d-
I1213 02:15:25.117481  8180 slave.cpp:6114] Shutting down executor 
'1537bacb-6889-48e5-a95d-02b62f947f6a' of framework 
c2c5066d-c610-4aa2-b10f-53065224bc8d- at executor(1)@10.3.1.5:58813
I1213 02:15:25.118502  8180 slave.cpp:909] Agent terminating
W1213 02:15:25.118502  8180 slave.cpp:3396] Ignoring shutdown framework 
c2c5066d-c610-4aa2-b10f-53065224bc8d- because it is terminating
I1213 02:15:25.119480   452 master.cpp:10264] Removing task 
1537bacb-6889-48e5-a95d-02b62f947f6a with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: *):[3I1213 
02:15:24.437484  5440 exec.cpp:162] Version: 1.5.0
I1213 02:15:24.460510  5040 exec.cpp:237] Executor registered on agent 
c2c5066d-c610-4aa2-b10f-53065224bc8d-S0
I1213 02:15:24.464485  8704 executor.cpp:171] Received SUBSCRIBED event
I1213 02:15:24.468502  8704 executor.cpp:175] Subscribed executor on 
build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1213 02:15:24.468502  8704 executor.cpp:171] Received LAUNCH event
I1213 02:15:24.472509  8704 executor.cpp:637] Starting task 
1537bacb-6889-48e5-a95d-02b62f947f6a
I1213 02:15:24.547503  8704 executor.cpp:477] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1213 02:15:25.087481  8704 executor.cpp:650] Forked command at 1760
I1213 02:15:25.118502  4128 exec.cpp:435] Executor asked to shutdown
I1213 02:15:25.119480  7408 executor.cpp:171] Received SHUTDOWN event
I1213 02:15:25.119480  7408 executor.cpp:747] Shutting down
I1213 02:15:25.119480  7408 executor.cpp:854] Sending SIGTERM to process tree 
at pid 11000-32000] of framework c2c5066d-c610-4aa2-b10f-53065224bc8d- on 
agent c2c5066d-c610-4aa2-b10f-53065224bc8d-S0 at slave(326)@10.3.1.5:58792 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1213 02:15:25.121481  8180 containerizer.cpp:2328] Destroying container 
a36336a7-78e0-45af-8dba-6b85f887048d in RUNNING state
I1213 02:15:25.121481  8180 containerizer.cpp:2930] Transitioning the state of 
container a36336a7-78e0-45af-8dba-6b85f887048d from RUNNING to DESTROYING
I1213 02:15:25.121481   452 master.cpp:1305] Agent 
c2c5066d-c610-4aa2-b10f-53065224bc8d-S0 at slave(326)@10.3.1.5:58792 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1213

Review Request 64563: Moved a comment to a more relevant location.

2017-12-12 Thread Meng Zhu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64563/
---

Review request for mesos and Benjamin Mahler.


Repository: mesos


Description
---

Moved a comment to a more relevant location.


Diffs
-

  src/master/allocator/mesos/hierarchical.cpp 
2b2d1fd2802203eba482be2992a5f2756d100cbf 


Diff: https://reviews.apache.org/r/64563/diff/1/


Testing
---


Thanks,

Meng Zhu

Re: Review Request 64562: Moved a comment to a more relevant location.

2017-12-12 Thread Benjamin Mahler 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64562/#review193635
---


Ship it!




Ship It!

- Benjamin Mahler


On Dec. 13, 2017, 1:19 a.m., Meng Zhu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64562/
> ---
> 
> (Updated Dec. 13, 2017, 1:19 a.m.)
> 
> 
> Review request for mesos and Benjamin Mahler.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Moved a comment to a more relevant location.
> 
> 
> Diffs
> -
> 
>   src/master/allocator/mesos/hierarchical.cpp 
> 2b5f114ccc24d073731f9e9abb1e269f6720e84d 
> 
> 
> Diff: https://reviews.apache.org/r/64562/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Meng Zhu
> 
>

Re: Review Request 64387: Windows: Ported docker health check tests.

2017-12-12 Thread Akash Gupta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64387/
---

(Updated Dec. 13, 2017, 1:48 a.m.)


Review request for mesos, Andrew Schwartzmeyer, Jie Yu, Joseph Wu, and Michael 
Park.


Changes
---

Rebased + addressed Andy's feedback.


Repository: mesos


Description
---

The `HealthCheckTest.ROOT_DOCKER_*` and
`DockerContainerizerHealthCheckTest.*` tests now work on Windows.


Diffs (updated)
-

  src/tests/environment.cpp 72bd621f02f97ea5fd553f3dc0bd52adb8ddee8f 
  src/tests/health_check_tests.cpp bb909c5ea4a51b562ac753083fda002195ac08ad 
  src/tests/mesos.hpp 320e9755c73bcf19ba8ac87f2081f17fdaa438ea 


Diff: https://reviews.apache.org/r/64387/diff/2/

Changes: https://reviews.apache.org/r/64387/diff/1-2/


Testing (updated)
---

Windows Server:
[==] Running 5 tests from 2 test cases.
[--] Global test environment set-up.
[--] 2 tests from HealthCheckTest
[ RUN  ] HealthCheckTest.ROOT_DOCKER_DockerHealthyTask
[   OK ] HealthCheckTest.ROOT_DOCKER_DockerHealthyTask (21263 ms)
[ RUN  ] HealthCheckTest.ROOT_DOCKER_DockerHealthStatusChange
[   OK ] HealthCheckTest.ROOT_DOCKER_DockerHealthStatusChange (23512 ms)
[--] 2 tests from HealthCheckTest (44835 ms total)

[--] 3 tests from NetworkProtocol/DockerContainerizerHealthCheckTest
[ RUN  ] 
NetworkProtocol/DockerContainerizerHealthCheckTest.ROOT_DOCKER_USERNETWORK_NETNAMESPACE_HealthyTaskViaHTTP/0
[   OK ] 
NetworkProtocol/DockerContainerizerHealthCheckTest.ROOT_DOCKER_USERNETWORK_NETNAMESPACE_HealthyTaskViaHTTP/0
 (28487 ms)
[ RUN  ] 
NetworkProtocol/DockerContainerizerHealthCheckTest.ROOT_DOCKER_USERNETWORK_NETNAMESPACE_HealthyTaskViaHTTPS/0
[   OK ] 
NetworkProtocol/DockerContainerizerHealthCheckTest.ROOT_DOCKER_USERNETWORK_NETNAMESPACE_HealthyTaskViaHTTPS/0
 (26447 ms)
[ RUN  ] 
NetworkProtocol/DockerContainerizerHealthCheckTest.ROOT_DOCKER_USERNETWORK_NETNAMESPACE_HealthyTaskViaTCP/0
[   OK ] 
NetworkProtocol/DockerContainerizerHealthCheckTest.ROOT_DOCKER_USERNETWORK_NETNAMESPACE_HealthyTaskViaTCP/0
 (26264 ms)
[--] 3 tests from NetworkProtocol/DockerContainerizerHealthCheckTest 
(81268 ms total)

[--] Global test environment tear-down
[==] 5 tests from 2 test cases ran. (126559 ms total)
[  PASSED  ] 5 tests

Rest of tests pass.

Windows Client (Disabled network health checks):
Proof that network health checks are skipped on Windows Client.
C:\Program Files\Docker\Docker\Resources\bin\docker.exe: Error response from 
daemon: sharing of hyperv containers network is not supported.
356b087e7fa640f83fe27ebeb3396bfc7b2bbebd917aeaec0508b887b41d31f4
-
We cannot run any Docker health checks tests because:
Running in another container's namespace is not supported on this version of 
Windows.

Rest rests pass.

Linux:
make check passes


Thanks,

Akash Gupta

Re: Review Request 64387: Windows: Ported docker health check tests.

2017-12-12 Thread Akash Gupta 


> On Dec. 7, 2017, 10:30 p.m., Andrew Schwartzmeyer wrote:
> > src/tests/health_check_tests.cpp
> > Lines 102-111 (patched)
> > 
> >
> > Can we reuse those added to the other test file?

Yeah. I can add the constant to `tests/mesos.hpp`, where the other global test 
constants are defined. However, I didn't make this patch dependent on the 
docker tests patch, so I think it's better to have another patch, so the change 
isn't duplicated.


> On Dec. 7, 2017, 10:30 p.m., Andrew Schwartzmeyer wrote:
> > src/tests/health_check_tests.cpp
> > Lines 110 (patched)
> > 
> >
> > Oh, oh! You're making a custom Docker image for this anyway. Add a 
> > symlink or something so you can just call `powershell.exe` to call 
> > `pwsh.exe`. Then some code disappears.

Unfortunately that doesn't work :(.


> On Dec. 7, 2017, 10:30 p.m., Andrew Schwartzmeyer wrote:
> > src/tests/health_check_tests.cpp
> > Lines 946 (patched)
> > 
> >
> > Probably not applicable when running in a container, but `-NoProfile` 
> > is generally recommended for scripted code.

`-NoProfile` doesn't exist in either in the container image or powershell core.


> On Dec. 7, 2017, 10:30 p.m., Andrew Schwartzmeyer wrote:
> > src/tests/health_check_tests.cpp
> > Lines 946-953 (patched)
> > 
> >
> > I've tried to consistently not use aliases (`New-Item -ItemType 
> > Directory` over `mkdir`) and use the right casing `Set-Content`. It's 
> > probably not important, but so long as I'm nitpicking I'll mention it. (And 
> > I don't necessarily agree with myself currently on not using `mkdir`).

I'll clean up the powershell script a bit.


> On Dec. 7, 2017, 10:30 p.m., Andrew Schwartzmeyer wrote:
> > src/tests/health_check_tests.cpp
> > Lines 948 (patched)
> > 
> >
> > _Could_ shorten it with `if (-Not (Remove-Item ... ))` rather than `$?`.

That changes the logic. It should alternate between creating the directory and 
not, which tests that health check going healthy <-> unhealthy. I based it off 
the bash code that was used on Linux.


> On Dec. 7, 2017, 10:30 p.m., Andrew Schwartzmeyer wrote:
> > src/tests/health_check_tests.cpp
> > Line 2227 (original), 2279-2282 (patched)
> > 
> >
> > Should we file a `TODO` issue to come back to this when IPv6 does work? 
> > I expect that's coming eventually.

Yeah. I expect it to some eventually.


> On Dec. 7, 2017, 10:30 p.m., Andrew Schwartzmeyer wrote:
> > src/tests/health_check_tests.cpp
> > Line 2249 (original), 2304-2307 (patched)
> > 
> >
> > Ditto, and also, would it make sense to make these functions a no-op on 
> > Windows for now so we don't have to worry about other code calling them?

Yeah. That's test code only, but it makes sense to me to have the `#ifdef` 
inside it so that it doesn't break Windows when other people use it.


> On Dec. 7, 2017, 10:30 p.m., Andrew Schwartzmeyer wrote:
> > src/tests/health_check_tests.cpp
> > Line 2258 (original), 2318-2324 (patched)
> > 
> >
> > Could probably not repeat some of this with just:
> > ```
> > DockerContainerizerHealthCheckTest,
> > ::testing::Values(
> > #ifdef __WINDOWS__
> > NetworkInfo::IPv4
> > #else
> > NetworkInfo::IPv4, NetworkInfo::IPv6
> > #endif
> > ));
> > ```
> > 
> > but it's a style choice and I know some other committers prefer the 
> > extra code over breaking up a piece of it.

Personally, having the `#ifdef` outside looks cleaner to me.


> On Dec. 7, 2017, 10:30 p.m., Andrew Schwartzmeyer wrote:
> > src/tests/health_check_tests.cpp
> > Lines 2562-2565 (original), 2647-2653 (patched)
> > 
> >
> > Is the same as the first one above?

Yeah, it's the same.


- Akash


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64387/#review193181
---


On Dec. 7, 2017, 10:30 p.m., Akash Gupta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64387/
> ---
> 
> (Updated Dec. 7, 2017, 10:30 p.m.)
> 
> 
> Review request for mesos, Andrew Schwartzmeyer, Jie Yu, Joseph Wu, and

Re: Review Request 64386: Windows: Enabled docker health checks.

2017-12-12 Thread Akash Gupta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64386/
---

(Updated Dec. 13, 2017, 1:43 a.m.)


Review request for mesos, Andrew Schwartzmeyer, Jie Yu, Joseph Wu, and Michael 
Park.


Changes
---

Fixed field reordering warning.


Repository: mesos


Description (updated)
---

The network health checks called curl and then executed setns to
enter to container network namespace, which did not work on Windows.
To do the equivalent, Windows nows calls docker run with powershell's
curl equivalent (Invoke-WebRequest) and uses the
network=container: flag to enter the container's namespace.


Diffs (updated)
-

  src/checks/checker.hpp 93502270f31e80c5f7c94b5b456625e9cdea1837 
  src/checks/checker.cpp fff0aac504b4283a210f936e00c977fa60d88b3d 
  src/checks/checker_process.hpp 510f3b2e6e689faaf26595214ce377c2b5518f28 
  src/checks/checker_process.cpp 496bd41811a1984e905c851b5e990495517c17b8 
  src/checks/health_checker.hpp 019fbd791f250ecc28ff59d779f90e7ccbf0c685 
  src/checks/health_checker.cpp eaf9a18817eeeff7c29c7a4b9d1b183f398760a3 
  src/docker/executor.cpp 3974f20052e3c12eb154a5146d19d4dc1759859f 
  src/launcher/executor.cpp c688c04e598ac140421fd5e47359b0e48d30bcc5 


Diff: https://reviews.apache.org/r/64386/diff/2/

Changes: https://reviews.apache.org/r/64386/diff/1-2/


Testing (updated)
---

See https://reviews.apache.org/r/64387/


Thanks,

Akash Gupta

Re: Review Request 64562: Moved a comment a more relevant location.

2017-12-12 Thread Meng Zhu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64562/
---

(Updated Dec. 12, 2017, 5:19 p.m.)


Review request for mesos and Benjamin Mahler.


Repository: mesos


Description (updated)
---

Moved a comment to a more relevant location.


Diffs
-

  src/master/allocator/mesos/hierarchical.cpp 
2b5f114ccc24d073731f9e9abb1e269f6720e84d 


Diff: https://reviews.apache.org/r/64562/diff/1/


Testing
---


Thanks,

Meng Zhu

Review Request 64562: Moved a comment a more relevant location.

2017-12-12 Thread Meng Zhu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64562/
---

Review request for mesos and Benjamin Mahler.


Repository: mesos


Description
---

Moved a comment a more relevant location.


Diffs
-

  src/master/allocator/mesos/hierarchical.cpp 
2b5f114ccc24d073731f9e9abb1e269f6720e84d 


Diff: https://reviews.apache.org/r/64562/diff/1/


Testing
---


Thanks,

Meng Zhu

Re: Review Request 64560: Fixed typo in agent.proto.

2017-12-12 Thread Greg Mann 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64560/#review193632
---


Ship it!




Ship It!

- Greg Mann


On Dec. 13, 2017, 12:41 a.m., Gaston Kleiman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64560/
> ---
> 
> (Updated Dec. 13, 2017, 12:41 a.m.)
> 
> 
> Review request for mesos, Chun-Hung Hsiao and Greg Mann.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Fixed typo in agent.proto.
> 
> 
> Diffs
> -
> 
>   include/mesos/agent/agent.proto 6fcca6a535f36c6ac4a82fcef31c719f4fbedb9d 
>   include/mesos/v1/agent/agent.proto 57c3518ce43bf7b003e0893ec2530f3091915d73 
> 
> 
> Diff: https://reviews.apache.org/r/64560/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Gaston Kleiman
> 
>

Re: Review Request 64543: Exposed resource provider information in master 'GET_AGENTS' response.

2017-12-12 Thread Jie Yu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64543/#review193631
---


Ship it!




Ship It!

- Jie Yu


On Dec. 12, 2017, 2:38 p.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64543/
> ---
> 
> (Updated Dec. 12, 2017, 2:38 p.m.)
> 
> 
> Review request for mesos, Jie Yu and Jan Schlicht.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch extends the master's 'GET_AGENTS' response to include local
> resource provider information. For now only the 'ResourceProviderInfo'
> are shown.
> 
> 
> Diffs
> -
> 
>   include/mesos/master/master.proto 87083beec66a9f57803bfb0735188fcdedafaa87 
>   include/mesos/v1/master/master.proto 
> ea04b8f1e28e1c20ac803f8f1610cc1e765ddd9d 
>   src/master/http.cpp bb040ced65495cfe78b67ca7258499c16ac5d8a1 
>   src/tests/api_tests.cpp 3372849cb2aa070df6b2262b57baa0fab628e11d 
> 
> 
> Diff: https://reviews.apache.org/r/64543/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>

Review Request 64561: Removed resource categories in UpdateSlaveMessage.

2017-12-12 Thread Jie Yu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64561/
---

Review request for mesos, Benjamin Bannier and Jan Schlicht.


Repository: mesos


Description
---

Given that now we use `UpdateSlaveMessage` to send resource provider
information directly, having resource categories in the message is
unnecessary and misleading.

Instead, this patch introduced a single optional boolean to indicate if
oversubscribed resources need to be updated or not.


Diffs
-

  src/master/master.hpp 232cc3758f240db626c4fdaf852163fa48af4dd7 
  src/master/master.cpp efe8b8f1704b314e6e6a4d5632718cab2854e38f 
  src/messages/messages.proto e680cd5e4d5a93c3c77309f327844f55fbb239a1 
  src/slave/slave.hpp 7c40fc71b49057fea0cfd85290931fbd0f6a9d62 
  src/slave/slave.cpp 5869e73ca1c14c99e580da9d7375181da2073ec5 
  src/tests/oversubscription_tests.cpp 3f57ce105e24e9f9cd681d8d984dbe242aa51f75 


Diff: https://reviews.apache.org/r/64561/diff/1/


Testing
---

make check


Thanks,

Jie Yu

Re: Review Request 64557: Handled the RP disconnection case in the agent.

2017-12-12 Thread Mesos Reviewbot Windows 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64557/#review193630
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64557']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64557

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64557/logs/mesos-tests-stdout.log):

```
[ RUN  ] SlaveTest.ShutdownV0ExecutorIfItReregistersWithoutReconnect
[   OK ] SlaveTest.ShutdownV0ExecutorIfItReregistersWithoutReconnect (244 
ms)
[ RUN  ] SlaveTest.IgnoreV0ExecutorIfItReregistersWithoutReconnect
[   OK ] SlaveTest.IgnoreV0ExecutorIfItReregistersWithoutReconnect (250 ms)
[ RUN  ] SlaveTest.BrowseExecutorSandboxByVirtualPath
[   OK ] SlaveTest.BrowseExecutorSandboxByVirtualPath (289 ms)
[ RUN  ] SlaveTest.DisconnectedExecutorDropsMessages
[   OK ] SlaveTest.DisconnectedExecutorDropsMessages (276 ms)
[ RUN  ] SlaveTest.ResourceProviderSubscribe
[   OK ] SlaveTest.ResourceProviderSubscribe (209 ms)
[ RUN  ] SlaveTest.ResourceProviderPublishAll
D:\DCOS\mesos\mesos\src\tests\slave_tests.cpp(8915): error: Mock function 
called more times than expected - returning directly.
Function call: statusUpdate(01BC8156B210, @0034280FE440 136-byte 
object <40-88 12-83 F7-7F 00-00 00-00 00-00 00-00 00-00 BE-A8 00-00 00-00 00-00 
C8-08 9E-84 F7-7F 00-00 D0-62 EE-81 BC-01 00-00 10-76 EE-81 BC-01 00-00 80-B2 
A6-81 BC-01 00-00 80-AF A6-81 BC-01 00-00 ... 00-00 00-00 00-00 00-00 60-52 
80-81 BC-01 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 
00-00 00-00 84-10 90-D3 1D-8C D6-41 00-00 00-00 02-00 00-00 00-00 00-00 03-00 
00-00>)
 Expected: to be called twice
   Actual: called 3 times - over-saturated and active
D:\DCOS\mesos\mesos\src\tests\slave_tests.cpp(8891): error: Mock function 
called more times than expected - returning directly.
Function call: resourceOffers(01BC8156B210, @0034283FEE50 { 
160-byte object <00-7D 12-83 F7-7F 00-00 00-00 00-00 00-00 00-00 5F-00 00-00 
00-00 00-00 00-00 00-00 00-00 00-00 05-00 00-00 05-00 00-00 40-26 70-81 BC-01 
00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 ... 60-3A 70-81 BC-01 
00-00 80-4D 77-82 BC-01 00-00 40-4B 77-82 BC-01 00-00 60-58 77-82 BC-01 00-00 
20-52 EC-81 BC-01 00-00 00-00 00-00 00-00 00-00 00-4F 77-82 BC-01 00-00 00-00 
00-00 00-00 00-00> })
 Expected: to be called once
   Actual: called twice - over-saturated and active
D:\DCOS\mesos\mesos\src\tests\slave_tests.cpp(8934): error:   Expected: i + 
1
  Which is: 2
To be equal to: v1::Resources(publish->resources()).reservations().size()
  Which is: 1
[  FAILED  ] SlaveTest.ResourceProviderPublishAll (8113 ms)
[ RUN  ] SlaveTest.ResourceVersions
[   OK ] SlaveTest.ResourceVersions (165 ms)
[ RUN  ] SlaveTest.ReconfigurationPolicy
[   OK ] SlaveTest.ReconfigurationPolicy (246 ms)
[ RUN  ] SlaveTest.ResourceProviderReconciliation
```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64557/logs/mesos-tests-stderr.log):

```
@   7FF77F7C41B0  google::LogMessage::SendToLog
@   7FF77F7C3997  google::LogMessage::Flush
@   7FF77F7C54D1  google::LogMessageFatal::~LogMessageFatal
@   7FF77D6E09A4  
mesos::internal::slave::Slave::handleResourceProviderMessage
@   7FF77D88BDD5   ?? 
@   7FF77D779218  
std::_Invoker_functor::_Call<,process::Future,process::ProcessBase
 * __ptr64>
@   7FF77D7FEEE8  
std::invoke<,process::Future,process::ProcessBase
 * __ptr64>
@   7FF77D80F0BB  
lambda::internal::Partial<,process::Future,std::_Ph<1>
 
>::invoke_expand<,std::tuple
 >,process::ProcessBase * __ptr64>
@   7FF77D80528C  
std::invoke
 >,process::ProcessBase * __ptr64>
@   7FF77D75D311  
)
 >,process::ProcessBase * __ptr64
@   7FF77D899726  process::ProcessBase * 
__ptr64)>::CallableFn
 > >::operator(
@   7FF77F2B3BED  process::ProcessBase * __ptr64)>::operator(
@   7FF77F18CCC9  process::ProcessBase::consume
@   7FF77F307D4A  process::DispatchEvent::consume
@   7FF77B7633A7  process::ProcessBase::serve
@   7FF77F19A9AB  process::ProcessManager::resume
@   7FF77F2A4411   ?? 
@   7FF77F1E2EF0  
std::_Invoker_functor::_Call< >
@   7FF77F238890  std::invoke< 
>
@   7FF77F1F1CAC  
std::_LaunchPad,std::default_delete > 
>

Review Request 64560: Fixed typo in agent.proto.

2017-12-12 Thread Gaston Kleiman 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64560/
---

Review request for mesos, Chun-Hung Hsiao and Greg Mann.


Repository: mesos


Description
---

Fixed typo in agent.proto.


Diffs
-

  include/mesos/agent/agent.proto 6fcca6a535f36c6ac4a82fcef31c719f4fbedb9d 
  include/mesos/v1/agent/agent.proto 57c3518ce43bf7b003e0893ec2530f3091915d73 


Diff: https://reviews.apache.org/r/64560/diff/1/


Testing
---


Thanks,

Gaston Kleiman

Review Request 64558: Added a missing check for parent container has config recovered.

2017-12-12 Thread Zhitao Li 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64558/
---

Review request for mesos, Gilbert Song and Greg Mann.


Bugs: MESOS-8325
https://issues.apache.org/jira/browse/MESOS-8325


Repository: mesos


Description
---

Added a missing check for parent container has config recovered.


Diffs
-

  src/slave/containerizer/mesos/containerizer.cpp 
a67311734a7cd8155ba54fe9ac45b7a0e3312168 


Diff: https://reviews.apache.org/r/64558/diff/1/


Testing
---

make


Thanks,

Zhitao Li

Review Request 64559: Checkpointing `OfferOperation` in resource provider states.

2017-12-12 Thread Chun-Hung Hsiao 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64559/
---

Review request for mesos and Jie Yu.


Repository: mesos


Description
---

Instead of checkpointing `ApplyOfferOperation`, we now checkpoint
`OfferOperations` in resource provider states such that we can keep
track of completed operations as well.

This patch also does some code cleanup, and modifies a unit test for
storage local resource provider to issue operations in batches.


Diffs
-

  src/resource_provider/state.proto 321201e0aee9c104e831a6fa91f737216d8cd59a 
  src/resource_provider/storage/provider.cpp 
e806f44ef33405d4a2b133576c60be56e9fe3435 
  src/tests/storage_local_resource_provider_tests.cpp 
e5868bfcb71977d2b2a146865f12a5207010ae40 


Diff: https://reviews.apache.org/r/64559/diff/1/


Testing
---

sudo make check


Thanks,

Chun-Hung Hsiao

Re: Review Request 64475: Initialized offer operation status update manager in SLRP.

2017-12-12 Thread Chun-Hung Hsiao 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64475/
---

(Updated Dec. 13, 2017, 12:31 a.m.)


Review request for mesos, Gaston Kleiman, Greg Mann, and Jie Yu.


Changes
---

Addressed Gaston's comments.


Repository: mesos


Description
---

This patch adds an agent filesystem layout for checkpointing offer
operation status updates for resource providers, and initialized
a status update manager in storage local resource provider.


Diffs (updated)
-

  src/resource_provider/storage/provider.cpp 
e806f44ef33405d4a2b133576c60be56e9fe3435 
  src/slave/paths.hpp d645d871c36bbe8e766a98650f6aa23b6eab65d8 
  src/slave/paths.cpp b8004e76964abc210820368a89dbfa6928ef7bfd 


Diff: https://reviews.apache.org/r/64475/diff/3/

Changes: https://reviews.apache.org/r/64475/diff/2-3/


Testing
---

sudo make check


Thanks,

Chun-Hung Hsiao

Re: Review Request 64521: Made `StatusUpdateManagerProcess` fill in the latest status update.

2017-12-12 Thread Gaston Kleiman 


> On Dec. 12, 2017, 1:21 p.m., Greg Mann wrote:
> > src/status_update_manager/status_update_manager_process.hpp
> > Lines 513 (patched)
> > 
> >
> > AFAICT there's nothing preventing somebody from setting `latest_status` 
> > on an update they pass into this process, causing us to fail this check?

Added a validation in the SUM's `update` method.


- Gaston


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64521/#review193598
---


On Dec. 12, 2017, 4:25 p.m., Gaston Kleiman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64521/
> ---
> 
> (Updated Dec. 12, 2017, 4:25 p.m.)
> 
> 
> Review request for mesos, Chun-Hung Hsiao, Greg Mann, and Jie Yu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Made `StatusUpdateManagerProcess` fill in the latest status update.
> 
> 
> Diffs
> -
> 
>   src/status_update_manager/status_update_manager_process.hpp 
> 1ac64410594134ce3e709d396d0a74a3d0aa2007 
>   src/tests/offer_operation_status_update_manager_tests.cpp 
> a5327d3d070a390f2ceb918dc53a024fbfe8777e 
> 
> 
> Diff: https://reviews.apache.org/r/64521/diff/3/
> 
> 
> Testing
> ---
> 
> Added a new test, which passes on GNU/Linux. The other existing tests also 
> pass.
> 
> 
> Thanks,
> 
> Gaston Kleiman
> 
>

Re: Review Request 64521: Made `StatusUpdateManagerProcess` fill in the latest status update.

2017-12-12 Thread Gaston Kleiman 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64521/
---

(Updated Dec. 12, 2017, 4:25 p.m.)


Review request for mesos, Chun-Hung Hsiao, Greg Mann, and Jie Yu.


Changes
---

Addressed feedback.


Summary (updated)
-

Made `StatusUpdateManagerProcess` fill in the latest status update.


Repository: mesos


Description (updated)
---

Made `StatusUpdateManagerProcess` fill in the latest status update.


Diffs (updated)
-

  src/status_update_manager/status_update_manager_process.hpp 
1ac64410594134ce3e709d396d0a74a3d0aa2007 
  src/tests/offer_operation_status_update_manager_tests.cpp 
a5327d3d070a390f2ceb918dc53a024fbfe8777e 


Diff: https://reviews.apache.org/r/64521/diff/3/

Changes: https://reviews.apache.org/r/64521/diff/2-3/


Testing
---

Added a new test, which passes on GNU/Linux. The other existing tests also pass.


Thanks,

Gaston Kleiman

Re: Review Request 64556: Made master acknowledge unknown operations during agent reconciliation.

2017-12-12 Thread Mesos Reviewbot Windows 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64556/#review193627
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64504', '64505', '64556']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64556

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64556/logs/mesos-tests-stdout.log):

```

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2305 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2326 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2249 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2272 ms total)

[--] Global test environment tear-down
[==] 829 tests from 84 test cases ran. (307350 ms total)
[  PASSED  ] 819 tests.
[  FAILED  ] 10 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] SlaveTest.ResourceProviderPublishAll

10 FAILED TESTS
  YOU HAVE 204 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64556/logs/mesos-tests-stderr.log):

```
I1212 23:57:17.661146  2968 master.cpp:10180] Updating the state of task 
6370c5dd-2df9-4e98-9c6c-6803a1f9f420 of framework 
4e8d03ca-2776-40f0-87af-2d7c956024ea- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I1212 23:57:17.661146  6216 slave.cpp:3400] Shutting down framework 
4e8d03ca-2776-40f0-87af-2d7c956024ea-
I1212 23:57:17.662165  6216 slave.cpp:6091] Shutting down executor 
'6370c5dd-2df9-4e98-9c6c-6803a1f9f420' of framework 
4e8d03ca-2776-40f0-87af-2d7c956024ea- at executor(1)@10.3.1.5:54314
I1212 23:57:17.663148  6216 slave.cpp:909] Agent terminating
I1212 23:57:17.663148  2968 master.cpp:10286] Removing task 
6370c5dd-2df9-4e98-9c6c-6803a1f9f420 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework 4e8d03ca-2776-40f0-87af-2d7c956024ea- on 
agent 4e8d03ca-2776-40f0-87af-2d7c956024ea-S0 at slave(326)@10.3.1.5:54293 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1212 23:57:17.666147  8904 hierarchical.cpp:344] Removed framework 
4e8d03ca-2776-40f0-87af-2d7c956024ea-
W1212 23:57:17.666147  6216 slave.cpp:3396] Ignoring shutdown framework 
4e8d03ca-2776-40f0-87af-2d7c956024ea- because it is terminating
I1212 23:57:17.667157  2968 master.cpp:1305] Agent 
4e8d03ca-2776-40f0-87af-2d7c956024ea-S0 at slave(326)@10.3.I1212 
23:57:17.001132  4772 exec.cpp:162] Version: 1.5.0
I1212 23:57:17.024132  2876 exec.cpp:237] Executor registered on agent 
4e8d03ca-2776-40f0-87af-2d7c956024ea-S0
I1212 23:57:17.027130  2616 executor.cpp:171] Received SUBSCRIBED event
I1212 23:57:17.031128  2616 executor.cpp:175] Subscribed executor on 
build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1212 23:57:17.031128  2616 executor.cpp:171] Received LAUNCH event
I1212 23:57:17.035132  2616 executor.cpp:637] Starting task 
6370c5dd-2df9-4e98-9c6c-6803a1f9f420
I1212 23:57:17.112140  2616 executor.cpp:477] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1212 23:57:17.638144  2616 executor.cpp:650] Forked command at 7868
I1212 23:57:17.666147  6504 exec.cpp:435] Executor asked to shutdown
I1212 23:57:17.667157   880 executor.cpp:171] Received SHUTDOWN event
I1212 23:57:17.667157   880 executor.cpp:747] Shutting down
I1212 23:57:17.667157   880 executor.cpp:854] Sending SIGTERM to process tree 
at pid 71.5:54293 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1212 23:57:17.667157  2968 master.cpp:3364] Disconnecting agent 
4e8d03ca-2776-40f0-87af-2d7c956024ea-S0 at slave(326)@10.3.1.5:54293 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1212 23:57:17.667157  2968 master.cpp:3383] Deactivating

Re: Review Request 64522: WIP: try to make tests pass on Windows.

2017-12-12 Thread Mesos Reviewbot Windows 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64522/#review193622
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64517', '64472', '64473', '64509', '64521', '64522']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64522

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64522/logs/mesos-tests-stdout.log):

```
[ RUN  ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1
[   OK ] ContainerizerType/DefaultContainerDNSFlagTest.ValidateFlag/1 (37 
ms)
[--] 2 tests from ContainerizerType/DefaultContainerDNSFlagTest (69 ms 
total)

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2300 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2322 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2270 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2299 ms total)

[--] Global test environment tear-down
[==] 830 tests from 84 test cases ran. (309613 ms total)
[  PASSED  ] 823 tests.
[  FAILED  ] 7 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] SlaveTest.ResourceProviderPublishAll

 7 FAILED TESTS
  YOU HAVE 204 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64522/logs/mesos-tests-stderr.log):

```
I1212 23:07:08.811098  8412 slave.cpp:3400] Shutting down framework 
9d5c56a2-97da-4850-82c4-850eb9436c2d-
I1212 23:07:08.811098  8412 slave.cpp:6091] Shutting down executor 
'453234f7-65a1-483e-a46f-bcd8816085c9' of framework 
9d5c56a2-97da-4850-82c4-850eb9436c2d- at executor(1)@10.3.1.5:52424
I1212 23:07:08.811098  1652 master.cpp:10146] Updating the state of task 
453234f7-65a1-483e-a46f-bcd8816085c9 of framework 
9d5c56a2-97da-4850-82c4-850eb9436c2d- (latest state: TASK_KILLED, status 
update state: TAI1212 23:07:08.148102  4416 exec.cpp:162] Version: 1.5.0
I1212 23:07:08.172078  8400 exec.cpp:237] Executor registered on agent 
9d5c56a2-97da-4850-82c4-850eb9436c2d-S0
I1212 23:07:08.174113  3388 executor.cpp:171] Received SUBSCRIBED event
I1212 23:07:08.179103  3388 executor.cpp:175] Subscribed executor on 
build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1212 23:07:08.179103  3388 executor.cpp:171] Received LAUNCH event
I1212 23:07:08.183106  3388 executor.cpp:637] Starting task 
453234f7-65a1-483e-a46f-bcd8816085c9
I1212 23:07:08.257084  3388 executor.cpp:477] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1212 23:07:08.786097  3388 executor.cpp:650] Forked command at 5976
I1212 23:07:08.812098  4108 exec.cpp:435] Executor asked to shutdown
I1212 23:07:08.813099  7580 executor.cpp:171] Received SHUTDOWN event
I1212 23:07:08.813099  7580 executor.cpp:747] Shutting down
I1212 23:07:08.813099  7580 executor.cpp:854] Sending SIGTERM to process tree 
at pid 5SK_KILLED)
I1212 23:07:08.812098  8412 slave.cpp:909] Agent terminating
W1212 23:07:08.812098  8412 slave.cpp:3396] Ignoring shutdown framework 
9d5c56a2-97da-4850-82c4-850eb9436c2d- because it is terminating
I1212 23:07:08.814098  1652 master.cpp:10252] Removing task 
453234f7-65a1-483e-a46f-bcd8816085c9 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework 9d5c56a2-97da-4850-82c4-850eb9436c2d- on 
agent 9d5c56a2-97da-4850-82c4-850eb9436c2d-S0 at slave(326)@10.3.1.5:52403 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1212 23:07:08.815099  7796 containerizer.cpp:2328] Destroying container 
afbb257d-c221-4f84-8319-0bc5fe8ad210 in RUNNING state
I1212 23:07:08.815099  7796 containerizer.cpp:2930] Transitioning the state of 
container afbb257d-c221-4f84-8319-0bc5fe8ad210 from RUNNING to DESTROYING
I1212 23:07:08.816099  7796 launcher.cpp:156] Asked to destroy container 
afbb257d-c221-4f84-8319-0bc5fe8ad210
I1212 23:07:08.816099  1652 master.cpp:1305] Agent 
9d5c56a2-97da-4850-82c4-850eb9436c2d-S0 at slave(326)@10.3.1.5:52403

Review Request 64557: Handled the RP disconnection case in the agent.

2017-12-12 Thread Jie Yu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64557/
---

Review request for mesos, Benjamin Bannier, Chun-Hung Hsiao, and Greg Mann.


Repository: mesos


Description
---

If an RP is disconnected, we'll shrink its total resources to zero so
that no offer will be made on this RP until it reconnects. This prevents
frameworks from sending operations to the disconnected RP.


Diffs
-

  src/resource_provider/manager.cpp fd138b9914d925b5be7a11255dd632921c107dba 
  src/resource_provider/message.hpp eab90cffd6aab9e38207dcf109cc737171ed3953 
  src/slave/slave.cpp 5869e73ca1c14c99e580da9d7375181da2073ec5 
  src/tests/resource_provider_manager_tests.cpp 
e37a53ac6a03e2ea58dd6580fc8a399a1398d950 


Diff: https://reviews.apache.org/r/64557/diff/1/


Testing
---

make check


Thanks,

Jie Yu

Review Request 64556: Made master acknowledge unknown operations during agent reconciliation.

2017-12-12 Thread Greg Mann 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64556/
---

Review request for mesos, Benjamin Bannier, Gaston Kleiman, and Jie Yu.


Repository: mesos


Description
---

Made master acknowledge unknown operations during agent reconciliation.


Diffs
-

  src/master/master.cpp efe8b8f1704b314e6e6a4d5632718cab2854e38f 


Diff: https://reviews.apache.org/r/64556/diff/1/


Testing
---

make check


Thanks,

Greg Mann

Re: Review Request 64505: Made the agent check for offer operation update retries.

2017-12-12 Thread Mesos Reviewbot Windows 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64505/#review193613
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64504', '64505']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64505

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64505/logs/mesos-tests-stdout.log):

```

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2271 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2293 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2264 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2286 ms total)

[--] Global test environment tear-down
[==] 829 tests from 84 test cases ran. (307997 ms total)
[  PASSED  ] 819 tests.
[  FAILED  ] 10 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] SlaveTest.ResourceProviderPublishAll

10 FAILED TESTS
  YOU HAVE 204 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64505/logs/mesos-tests-stderr.log):

```
I1212 22:17:18.174181  7592 executor.cpp:171] Received SUBSCRIBED event
I1212 22:17:18.178182  7592 executor.cpp:175] Subscribed executor on 
build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1212 22:17:18.179183  7592 executor.cpp:171] Received LAUNCH event
I1212 22:17:18.182238  7592 executor.cpp:637] Starting task 
f6154322-8991-4748-aa62-3077933118d2
I1212 22:17:18.255249  7592 executor.cpp:477] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1212 22:17:18.778175  7592 executor.cpp:650] Forked command at 4108
I1212 22:17:18.803174  4760 exec.cpp:435] Executor asked to shutdown
I1212 22:17:18.804172  2264 executor.cpp:171] Received SHUTDOWN event
I1212 22:17:18.804172  2264 executor.cpp:747] Shutting down
I1212 22:17:18.804172  2264 executor.cpp:854] Sending SIGTERM to process tree 
at pid 41172  1536 master.cpp:3327] Deactivating framework 
c037d5f3-ab72-4df4-b368-8a2ad739ba7d- (default) at 
scheduler-6e9f27ea-7c29-4ead-80e1-71cce262173a@10.3.1.5:50490
I1212 22:17:18.802269  7016 hierarchical.cpp:405] Deactivated framework 
c037d5f3-ab72-4df4-b368-8a2ad739ba7d-
I1212 22:17:18.802269  1536 master.cpp:10146] Updating the state of task 
f6154322-8991-4748-aa62-3077933118d2 of framework 
c037d5f3-ab72-4df4-b368-8a2ad739ba7d- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I1212 22:17:18.802269  8568 slave.cpp:3400] Shutting down framework 
c037d5f3-ab72-4df4-b368-8a2ad739ba7d-
I1212 22:17:18.802269  8568 slave.cpp:6091] Shutting down executor 
'f6154322-8991-4748-aa62-3077933118d2' of framework 
c037d5f3-ab72-4df4-b368-8a2ad739ba7d- at executor(1)@10.3.1.5:50512
I1212 22:17:18.803174  6636 slave.cpp:909] Agent terminating
W1212 22:17:18.803174  6636 slave.cpp:3396] Ignoring shutdown framework 
c037d5f3-ab72-4df4-b368-8a2ad739ba7d- because it is terminating
I1212 22:17:18.804172  1536 master.cpp:10252] Removing task 
f6154322-8991-4748-aa62-3077933118d2 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework c037d5f3-ab72-4df4-b368-8a2ad739ba7d- on 
agent c037d5f3-ab72-4df4-b368-8a2ad739ba7d-S0 at slave(326)@10.3.1.5:50490 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1212 22:17:18.806172   444 containerizer.cpp:2328] Destroying container 
20fe4ee8-e897-43c6-b64f-0e5ee570cb1d in RUNNING state
I1212 22:17:18.806172   444 containerizer.cpp:2930] Transitioning the state of 
container 20fe4ee8-e897-43c6-b64f-0e5ee570cb1d from RUNNING to DESTROYING
I1212 22:17:18.807173  1536 master.cpp:1305] Agent 
c037d5f3-ab72-4df4-b368-8a2ad739ba7d-S0 at slave(326)@10.3.1.5:50490

Re: Review Request 64493: Added tests for quota enforcement with unallocated reservations.

2017-12-12 Thread Meng Zhu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64493/
---

(Updated Dec. 12, 2017, 1:45 p.m.)


Review request for mesos, Benjamin Mahler and Michael Park.


Changes
---

Thanks! Patch updated based on the comments.


Repository: mesos


Description
---

Added tests for quota enforcement with unallocated reservations.


Diffs (updated)
-

  src/tests/hierarchical_allocator_tests.cpp 
862f4683da04d37d9fe9f471d6ec9cd7751f39ec 


Diff: https://reviews.apache.org/r/64493/diff/2/

Changes: https://reviews.apache.org/r/64493/diff/1-2/


Testing
---

maek check


Thanks,

Meng Zhu

Re: Review Request 64493: Added tests for quota enforcement with unallocated reservations.

2017-12-12 Thread Meng Zhu 


> On Dec. 11, 2017, 6:07 p.m., Benjamin Mahler wrote:
> > src/tests/hierarchical_allocator_tests.cpp
> > Lines 1274-1280 (patched)
> > 
> >
> > Just curious, is it possible to template it by the 
> > `Resource::ReservationInfo::Type` enum?
> > 
> > That might read a little more easily in the test output and code.
> > 
> > In terms of naming, I think we tend to avoid naming these 
> > `SomeTestWithParam` (I only see the one named like this in this file). 
> > Naming this way means that if we add another parameterized test, we then 
> > need to add what it is parameterized by in the name:
> > 
> > ```
> > // Before:
> > HierarchicalAllocatorTestWithParam ... bool (for quota)
> > 
> > HierarchicalAllocatorTestWithQuotaParam ... bool (for quota)
> > HierarchicalAllocatorTestWithReservationParam ... bool or 
> > Resource::ReservationInfo::Type (for reservation)
> > ```
> > 
> > Then I think it can get confusing because one's test interpretation of 
> > a "quota parameter" might be different than another's and you may then need 
> > to split the names further to distinguish?
> > 
> > Maybe we could just name this fixture 
> > `HierarchicalAllocatorTestLimitWithReservations` and have the following?
> > 
> > ```
> > TEST_P(HierarchicalAllocatorLimitWithReservationsTest, Unallocated)
> > TEST_P(HierarchicalAllocatorTestWithReservationParam, Allocated)
> > ```

Good suggestion. Done.


> On Dec. 11, 2017, 6:07 p.m., Benjamin Mahler wrote:
> > src/tests/hierarchical_allocator_tests.cpp
> > Lines 1314-1319 (patched)
> > 
> >
> > Is it possible just to inline this?
> > 
> > ```
> > AWAIT_READY(allocator->updateAvailable(agent1.id(), 
> > {RESERVE(reserved)}));
> > ```

Done.


> On Dec. 11, 2017, 6:07 p.m., Benjamin Mahler wrote:
> > src/tests/hierarchical_allocator_tests.cpp
> > Lines 1323 (patched)
> > 
> >
> > Can you stringify QUOTA_ROLE instead of burning it in here?
> > 
> > Alternatively, I think you can pass `Resources` in?
> > 
> > ```
> > Resources resources = Resources::parse("cpus:1;mem:1024).get();
> > 
> > resources.push_reservation(a static reservation)
> > 
> > agent1 = createSlaveInfo(resources);
> > ```
> > 
> > Hm.. this makes me wonder why we couldn't just do the following with 
> > the parameter?
> > 
> > ```
> > Resource::ReservationInfo reservation;
> > reservation.set_type(GetParam());
> > reservation.set_role(QUOTA_ROLE);
> > 
> > Resources resources = Resources::parse("cpus:1;mem:1024").get();
> > resources = resources.pushReservation(reservation);
> > 
> > agent1 = createSlaveInfo(resources);
> > 
> > ...
> > ```
> > 
> > No need for an if condition?

Great suggestion! Done, much concise.


> On Dec. 11, 2017, 6:07 p.m., Benjamin Mahler wrote:
> > src/tests/hierarchical_allocator_tests.cpp
> > Lines 1471-1473 (patched)
> > 
> >
> > Hm.. what is this testing? I was expecting to see that you couldn't go 
> > over your limit when you have [un]allocated resesrvations. This test being 
> > the allocated case, the above test being the unallocate case?

The first test checks you couldn't go over your limit when you have unallocated 
reservation.
The second test checks that you can properly allocated up to your limit when 
you have partial reservation.
The case of all allocated reservation should be no different from the case 
where there is no reservations in the sense that they are just allocated 
resources. There are already test for that.

The second test is needed to check against the tricky case of double counting 
of allocated-reservations. (Hence the comment above the second test).


- Meng


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64493/#review193481
---


On Dec. 12, 2017, 1:45 p.m., Meng Zhu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64493/
> ---
> 
> (Updated Dec. 12, 2017, 1:45 p.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Michael Park.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added tests for quota enforcement with unallocated reservations.
> 
> 
> Diffs
> -
> 
>   src/tests/hierarchical_allocator_tests.cpp 
> 862f4683da04d37d9fe9f471d6ec9cd7751f39ec 
> 
> 
> Diff:

Re: Review Request 64505: Made the agent check for offer operation update retries.

2017-12-12 Thread Greg Mann 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64505/
---

(Updated Dec. 12, 2017, 9:22 p.m.)


Review request for mesos, Benjamin Bannier, Jie Yu, and Jan Schlicht.


Repository: mesos


Description
---

Local resource providers send all offer operation status updates
using the reosurce provider API. This patch makes the agent's
resource provider message handler skip operation updates when an
update is a retry.


Diffs (updated)
-

  src/slave/slave.cpp 5869e73ca1c14c99e580da9d7375181da2073ec5 


Diff: https://reviews.apache.org/r/64505/diff/2/

Changes: https://reviews.apache.org/r/64505/diff/1-2/


Testing
---

make check


Thanks,

Greg Mann

Re: Review Request 64521: Made `StatusUpdateManagerProcess` fill-in the latest status update.

2017-12-12 Thread Greg Mann 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64521/#review193598
---




src/status_update_manager/status_update_manager_process.hpp
Lines 513-514 (original), 511-514 (patched)


Let's add a `CHECK_NOTNULL(stream);` here as well.



src/status_update_manager/status_update_manager_process.hpp
Lines 513 (patched)


AFAICT there's nothing preventing somebody from setting `latest_status` on 
an update they pass into this process, causing us to fail this check?



src/tests/offer_operation_status_update_manager_tests.cpp
Lines 906-916 (patched)


Since we test this functionality elsewhere, we can remove this block.


- Greg Mann


On Dec. 12, 2017, 1:47 a.m., Gaston Kleiman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64521/
> ---
> 
> (Updated Dec. 12, 2017, 1:47 a.m.)
> 
> 
> Review request for mesos, Chun-Hung Hsiao, Greg Mann, and Jie Yu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Made `StatusUpdateManagerProcess` fill-in the latest status update.
> 
> 
> Diffs
> -
> 
>   src/status_update_manager/status_update_manager_process.hpp 
> 1ac64410594134ce3e709d396d0a74a3d0aa2007 
>   src/tests/offer_operation_status_update_manager_tests.cpp 
> a5327d3d070a390f2ceb918dc53a024fbfe8777e 
> 
> 
> Diff: https://reviews.apache.org/r/64521/diff/2/
> 
> 
> Testing
> ---
> 
> Added a new test, which passes on GNU/Linux. The other existing tests also 
> pass.
> 
> 
> Thanks,
> 
> Gaston Kleiman
> 
>

Re: Review Request 64522: WIP: try to make tests pass on Windows.

2017-12-12 Thread Mesos Reviewbot Windows 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64522/#review193604
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64517', '64472', '64473', '64509', '64521', '64522']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64522

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64522/logs/mesos-tests-stdout.log):

```
[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2359 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2387 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2367 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2392 ms total)

[--] Global test environment tear-down
[==] 830 tests from 84 test cases ran. (323756 ms total)
[  PASSED  ] 819 tests.
[  FAILED  ] 11 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateLatestWhenResending
[  FAILED  ] SlaveTest.ResourceProviderPublishAll

11 FAILED TESTS
  YOU HAVE 201 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64522/logs/mesos-tests-stderr.log):

```
I1212 20:41:19.765161  2624 exec.cpp:237] Executor registered on agent 
a2ed4f13-8018-4f90-820a-ffac2ca4629f-S0
I1212 20:41:19.768159  3512 executor.cpp:171] Received SUBSCRIBED event
I1212 20:41:19.772159  3512 executor.cpp:175] Subscribed executor on 
build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1212 20:41:19.772159  3512 executor.cpp:171] Received LAUNCH event
I1212 20:41:19.776159  3512 executor.cpp:637] Starting task 
9ffa43e5-8542-437a-a550-43b31f2e4d99
I1212 20:41:19.856178  3512 executor.cpp:477] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1212 20:41:20.427151  3512 executor.cpp:650] Forked command at 1944
I1212 20:41:20.453151  5912 exec.cpp:435] Executor asked to shutdown
I1212 20:41:20.454150  3512 executor.cpp:171] Received SHUTDOWN event
I1212 20:41:20.454150  3512 executor.cpp:747] Shutting down
I1212 20:41:20.454150  3512 executor.cpp:854] Sending SIGTERM to process tree 
at pid 1erarchical.cpp:405] Deactivated framework 
a2ed4f13-8018-4f90-820a-ffac2ca4629f-
I1212 20:41:20.451151   796 master.cpp:10146] Updating the state of task 
9ffa43e5-8542-437a-a550-43b31f2e4d99 of framework 
a2ed4f13-8018-4f90-820a-ffac2ca4629f- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I1212 20:41:20.451151 10068 slave.cpp:3400] Shutting down framework 
a2ed4f13-8018-4f90-820a-ffac2ca4629f-
I1212 20:41:20.452152 10068 slave.cpp:6091] Shutting down executor 
'9ffa43e5-8542-437a-a550-43b31f2e4d99' of framework 
a2ed4f13-8018-4f90-820a-ffac2ca4629f- at executor(1)@10.3.1.11:53409
I1212 20:41:20.453151 10068 slave.cpp:909] Agent terminating
W1212 20:41:20.453151 10068 slave.cpp:3396] Ignoring shutdown framework 
a2ed4f13-8018-4f90-820a-ffac2ca4629f- because it is terminating
I1212 20:41:20.454150   796 master.cpp:10252] Removing task 
9ffa43e5-8542-437a-a550-43b31f2e4d99 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework a2ed4f13-8018-4f90-820a-ffac2ca4629f- on 
agent a2ed4f13-8018-4f90-820a-ffac2ca4629f-S0 at slave(326)@10.3.1.11:53388 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1212 20:41:20.455150  5636 containerizer.cpp:2328] Destroying container 
d711ad02-00d5-49c2-a2be-46d32d37bf3b in RUNNING state
I1212 20:41:20.456151  5636 containerizer.cpp:2930] Transitioning the state of 
container d711ad02-00d5-49c2-a2be-46d32d37bf3b from RUNNING to DESTROYING
I1212 20:41:20.456151  5636 launcher.cpp:156] Asked to destroy container 
d711ad02-00d5-49c2-a2be-46d32d37bf3b
I1212 20:41:20.457150   796 master.cpp:1305]

Re: Review Request 64548: Made the agent respond to reconciliation requests for default resources.

2017-12-12 Thread Gaston Kleiman 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64548/#review193601
---


Ship it!




Ship It!

- Gaston Kleiman


On Dec. 12, 2017, 10:21 a.m., Greg Mann wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64548/
> ---
> 
> (Updated Dec. 12, 2017, 10:21 a.m.)
> 
> 
> Review request for mesos, Benjamin Bannier, Gaston Kleiman, Jie Yu, and Jan 
> Schlicht.
> 
> 
> Bugs: MESOS-8195
> https://issues.apache.org/jira/browse/MESOS-8195
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Made the agent respond to reconciliation requests for default resources.
> 
> 
> Diffs
> -
> 
>   src/slave/slave.cpp 5869e73ca1c14c99e580da9d7375181da2073ec5 
> 
> 
> Diff: https://reviews.apache.org/r/64548/diff/1/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Greg Mann
> 
>

Re: Review Request 64551: Integrated the status update manager into SLRP.

2017-12-12 Thread Gaston Kleiman 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64551/#review193594
---



In the commit message: `s/and garbage collect/and garbage collects/`


src/resource_provider/storage/provider.cpp
Lines 904-905 (patched)


Why can't we reconstruct them?

The list of status returned by the SUM will contain a terminal update, but 
`terminated` will be `false`.



src/resource_provider/storage/provider.cpp
Lines 2538 (patched)


s/refactor/refactoring/


- Gaston Kleiman


On Dec. 12, 2017, 9:52 a.m., Chun-Hung Hsiao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64551/
> ---
> 
> (Updated Dec. 12, 2017, 9:52 a.m.)
> 
> 
> Review request for mesos, Gaston Kleiman, Greg Mann, and Jie Yu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch uses the status update manager to send status updates and
> recover operations that are checkpointed as completed in the status
> update manager but still in the pending list in SLRP. It also forwards
> the acknowledgements to the status update manager, and garbage collect
> the metadata for offer operations.
> 
> 
> Diffs
> -
> 
>   src/resource_provider/daemon.hpp 7c513a2b7c7ca958f9ea57326493f74df5b67c8f 
>   src/resource_provider/daemon.cpp 7c783e309672a2d93532fb17f13afcb96d5a7006 
>   src/resource_provider/local.hpp 46111d9f8969ea47e695d08832cb5a9ea60d1971 
>   src/resource_provider/local.cpp d1d683534d186e77a2beb32c978f2aaea289b086 
>   src/resource_provider/storage/provider.hpp 
> 374f837ad693820afffb86bd1e54915f6796d088 
>   src/resource_provider/storage/provider.cpp 
> e806f44ef33405d4a2b133576c60be56e9fe3435 
>   src/slave/paths.hpp d645d871c36bbe8e766a98650f6aa23b6eab65d8 
>   src/slave/paths.cpp b8004e76964abc210820368a89dbfa6928ef7bfd 
> 
> 
> Diff: https://reviews.apache.org/r/64551/diff/1/
> 
> 
> Testing
> ---
> 
> sudo make check
> 
> 
> Thanks,
> 
> Chun-Hung Hsiao
> 
>

Re: Review Request 64548: Made the agent respond to reconciliation requests for default resources.

2017-12-12 Thread Mesos Reviewbot Windows 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64548/#review193597
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64548']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64548

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64548/logs/mesos-tests-stdout.log):

```

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2449 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2472 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2378 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2400 ms total)

[--] Global test environment tear-down
[==] 829 tests from 84 test cases ran. (328217 ms total)
[  PASSED  ] 819 tests.
[  FAILED  ] 10 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] SlaveTest.ResourceProviderPublishAll

10 FAILED TESTS
  YOU HAVE 201 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64548/logs/mesos-tests-stderr.log):

```
I1212 19:41:58.563194  9360 master.cpp:10146] Updating the state of task 
13559cc4-0fdf-474c-9b0f-1fcb9b93ad97 of framework 
fa6537db-6e21-4f20-9b03-21663f4af419- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I1212 19:41:58.563194  3712 slave.cpp:3400] Shutting down framework 
fa6537db-6e21-4f20-9b03-21663f4af419-
I1212 19:41:58.564195  3712 slave.cpp:6114] Shutting down executor 
'13559cc4-0fdf-474c-9b0f-1fcb9b93ad97' of framework 
fa6537db-6e21-4f20-9b03-21663f4af419- at executor(1)@10.3.1.11:51505
I1212 19:41:58.564195  3712 slave.cpp:909] Agent terminating
W1212 19:41:58.565196 I1212 19:41:57.891209  7284 exec.cpp:162] Version: 1.5.0
I1212 19:41:57.914206  8292 exec.cpp:237] Executor registered on agent 
fa6537db-6e21-4f20-9b03-21663f4af419-S0
I1212 19:41:57.918203  5032 executor.cpp:171] Received SUBSCRIBED event
I1212 19:41:57.922207  5032 executor.cpp:175] Subscribed executor on 
build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1212 19:41:57.922207  5032 executor.cpp:171] Received LAUNCH event
I1212 19:41:57.927207  5032 executor.cpp:637] Starting task 
13559cc4-0fdf-474c-9b0f-1fcb9b93ad97
I1212 19:41:58.003206  5032 executor.cpp:477] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1212 19:41:58.537196  5032 executor.cpp:650] Forked command at 5180
I1212 19:41:58.565196  7944 exec.cpp:435] Executor asked to shutdown
I1212 19:41:58.566195  5032 executor.cpp:171] Received SHUTDOWN event
I1212 19:41:58.566195  5032 executor.cpp:747] Shutting down
I1212 19:41:58.566195  5032 executor.cpp:854] Sending SIGTERM to process tree 
at pid 5 3712 slave.cpp:3396] Ignoring shutdown framework 
fa6537db-6e21-4f20-9b03-21663f4af419- because it is terminating
I1212 19:41:58.565196  9360 master.cpp:10252] Removing task 
13559cc4-0fdf-474c-9b0f-1fcb9b93ad97 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework fa6537db-6e21-4f20-9b03-21663f4af419- on 
agent fa6537db-6e21-4f20-9b03-21663f4af419-S0 at slave(326)@10.3.1.11:51483 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1212 19:41:58.568195  3712 containerizer.cpp:2328] Destroying container 
d6e1e150-3a67-41fe-88d9-0ea0f1684305 in RUNNING state
I1212 19:41:58.568195  9360 master.cpp:1305] Agent 
fa6537db-6e21-4f20-9b03-21663f4af419-S0 at slave(326)@10.3.1.11:51483 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1212 19:41:58.568195  3712 containerizer.cpp:2930] Transitioning the state of 
container d6e1e150-3a67-41fe-88d9-0ea0f1684305 from RUNNING to DESTROYING
I1212 19:41:58.568195  9360 master.cpp:3364] Disconnecting agent 
fa6537db-6e21-4f20-9b03-21663f4af419-S0

Re: Review Request 64551: Integrated the status update manager into SLRP.

2017-12-12 Thread Mesos Reviewbot Windows 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64551/#review193596
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64517', '64472', '64473', '64509', '64521', '64550', 
'64475', '64551']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64551

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64551/logs/mesos-tests-stdout.log):

```
[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2283 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2305 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2266 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2289 ms total)

[--] Global test environment tear-down
[==] 830 tests from 84 test cases ran. (301567 ms total)
[  PASSED  ] 819 tests.
[  FAILED  ] 11 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateLatestWhenResending
[  FAILED  ] SlaveTest.ResourceProviderPublishAll

11 FAILED TESTS
  YOU HAVE 204 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64551/logs/mesos-tests-stderr.log):

```
I1212 19:37:06.878981  9156 master.cpp:10146] Updating the state of task 
649405f8-356f-435b-ab7a-d412c08c11e9 of framework 
14ee5ed0-9a4a-43a8-886b-91025859d73b- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I1212 19:37:06.878981  3548 slave.cpp:3400] Shutting down framework 
14ee5ed0-9a4a-43a8-886b-91025859d73b-
I1212 19:37:06.878981  3548 slave.cpp:6091] Shutting down executor 
'649405f8-356f-435b-ab7a-d412c08c11e9' of framework 
14ee5ed0-9a4a-43a8-886b-91025859d73b- at executor(1)@10.I1212 
19:37:06.221943  5104 exec.cpp:162] Version: 1.5.0
I1212 19:37:06.244920  4568 exec.cpp:237] Executor registered on agent 
14ee5ed0-9a4a-43a8-886b-91025859d73b-S0
I1212 19:37:06.247920  8728 executor.cpp:171] Received SUBSCRIBED event
I1212 19:37:06.251940  8728 executor.cpp:175] Subscribed executor on 
build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1212 19:37:06.252938  8728 executor.cpp:171] Received LAUNCH event
I1212 19:37:06.255962  8728 executor.cpp:637] Starting task 
649405f8-356f-435b-ab7a-d412c08c11e9
I1212 19:37:06.329921  8728 executor.cpp:477] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1212 19:37:06.853941  8728 executor.cpp:650] Forked command at 6188
I1212 19:37:06.880908  3324 exec.cpp:435] Executor asked to shutdown
I1212 19:37:06.880908  8252 executor.cpp:171] Received SHUTDOWN event
I1212 19:37:06.880908  8252 executor.cpp:747] Shutting down
I1212 19:37:06.881907  8252 executor.cpp:854] Sending SIGTERM to process tree 
at pid 63.1.5:64835
I1212 19:37:06.879907  3548 slave.cpp:909] Agent terminating
W1212 19:37:06.879907  3548 slave.cpp:3396] Ignoring shutdown framework 
14ee5ed0-9a4a-43a8-886b-91025859d73b- because it is terminating
I1212 19:37:06.880908  9156 master.cpp:10252] Removing task 
649405f8-356f-435b-ab7a-d412c08c11e9 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework 14ee5ed0-9a4a-43a8-886b-91025859d73b- on 
agent 14ee5ed0-9a4a-43a8-886b-91025859d73b-S0 at slave(326)@10.3.1.5:64814 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1212 19:37:06.883906  9156 master.cpp:1305] Agent 
14ee5ed0-9a4a-43a8-886b-91025859d73b-S0 at slave(326)@10.3.1.5:64814 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1212 19:37:06.883906  9156 master.cpp:3364] Disconnecting agent 
14ee5ed0-9a4a-43a8-886b-91025859d73b-S0 at slave(326)@10.3.1.5:64814 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1212 19:37:06.883906  8108

Re: Review Request 64550: Made `update` return a future completed before `forwardCallback`.

2017-12-12 Thread Gaston Kleiman 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64550/#review193593
---


Fix it, then Ship it!




In the commit message: `s/This is useful if the caller of the status update 
manager want/This is useful if
the caller of the status update manager wants/`


src/status_update_manager/status_update_manager_process.hpp
Lines 513 (patched)


s/asynchoronus/asynchronous/

I'd put this comment right above the `dispatch` call.


- Gaston Kleiman


On Dec. 12, 2017, 9:49 a.m., Chun-Hung Hsiao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64550/
> ---
> 
> (Updated Dec. 12, 2017, 9:49 a.m.)
> 
> 
> Review request for mesos, Gaston Kleiman, Greg Mann, and Jie Yu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> We maintain an ordering invariant that the completion of the future
> returned by `update` happens before the invocation of `forwardCallback`.
> This invariant provides the guarantee that if `forwardCallback` is
> deferred to the same context as the chained callback of the returned
> future, the latter will happen before the earlier. This is useful if
> the caller of the status update manager want to maintain its internal
> state after the manager checkpoints the status update but before the
> status update is sent out.
> 
> 
> Diffs
> -
> 
>   src/status_update_manager/status_update_manager_process.hpp 
> 1ac64410594134ce3e709d396d0a74a3d0aa2007 
> 
> 
> Diff: https://reviews.apache.org/r/64550/diff/1/
> 
> 
> Testing
> ---
> 
> sudo make check
> 
> 
> Thanks,
> 
> Chun-Hung Hsiao
> 
>

Re: Review Request 64475: Initialized offer operation status update manager in SLRP.

2017-12-12 Thread Gaston Kleiman 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64475/#review193587
---




src/resource_provider/storage/provider.cpp
Line 362 (original), 365 (patched)


s/synchoronusly/synchronously/



src/resource_provider/storage/provider.cpp
Lines 2476-2477 (patched)


This loses the error message returned by `slave::state::checkpoint()`.

I think that the following would be make debugging easier:

```
  Try result = slave::state::checkpoint(
  statePath, volumes.at(volumeId).state);

  CHECK_SOME(result) << "Failed to checkpoint volume state to '" << 
statePath
 << "': " << result.error();
```



src/resource_provider/storage/provider.cpp
Lines 2445-2447 (original), 2492-2494 (patched)


```
// TODO(chhsiao): Maintain a list of terminated but unacknowledged
// offer operations in memory and reconstruct it during recovery
// by querying the status update manager.
```



src/resource_provider/storage/provider.cpp
Lines 2541-2543 (patched)


Once the following patch is committed, `statusUpdate.has_latest_status()` 
will always return `true`, so we don't the if statement: 
https://reviews.apache.org/r/64521/


- Gaston Kleiman


On Dec. 12, 2017, 9:51 a.m., Chun-Hung Hsiao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64475/
> ---
> 
> (Updated Dec. 12, 2017, 9:51 a.m.)
> 
> 
> Review request for mesos, Gaston Kleiman, Greg Mann, and Jie Yu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch adds an agent filesystem layout for checkpointing offer
> operation status updates for resource providers, and initialized
> a status update manager in storage local resource provider.
> 
> 
> Diffs
> -
> 
>   src/resource_provider/storage/provider.cpp 
> e806f44ef33405d4a2b133576c60be56e9fe3435 
>   src/slave/paths.hpp d645d871c36bbe8e766a98650f6aa23b6eab65d8 
>   src/slave/paths.cpp b8004e76964abc210820368a89dbfa6928ef7bfd 
> 
> 
> Diff: https://reviews.apache.org/r/64475/diff/2/
> 
> 
> Testing
> ---
> 
> sudo make check
> 
> 
> Thanks,
> 
> Chun-Hung Hsiao
> 
>

Re: Review Request 64550: Made `update` return a future completed before `forwardCallback`.

2017-12-12 Thread Mesos Reviewbot Windows 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64550/#review193586
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64517', '64472', '64473', '64509', '64521', '64550']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64550

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64550/logs/mesos-tests-stdout.log):

```
[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2412 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2437 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (3139 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (3162 ms total)

[--] Global test environment tear-down
[==] 830 tests from 84 test cases ran. (322308 ms total)
[  PASSED  ] 819 tests.
[  FAILED  ] 11 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateLatestWhenResending
[  FAILED  ] SlaveTest.ResourceProviderPublishAll

11 FAILED TESTS
  YOU HAVE 201 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64550/logs/mesos-tests-stderr.log):

```
I1212 18:41:56.392915  9544 master.cpp:10114] Updating the state of task 
d2e84fc2-29fe-4a26-8ad3-aae3fca54755 of framework 
eb15fa9d-8a4c-4edb-a501-24251020b202- (latest state: TASK_KILLED,I1212 
18:41:55.656930  7864 exec.cpp:162] Version: 1.5.0
I1212 18:41:55.680415   592 exec.cpp:237] Executor registered on agent 
eb15fa9d-8a4c-4edb-a501-24251020b202-S0
I1212 18:41:55.682927  5784 executor.cpp:171] Received SUBSCRIBED event
I1212 18:41:55.687929  5784 executor.cpp:175] Subscribed executor on 
build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1212 18:41:55.687929  5784 executor.cpp:171] Received LAUNCH event
I1212 18:41:55.689929  5784 executor.cpp:637] Starting task 
d2e84fc2-29fe-4a26-8ad3-aae3fca54755
I1212 18:41:55.774929  5784 executor.cpp:477] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1212 18:41:56.365924  5784 executor.cpp:650] Forked command at 5840
I1212 18:41:56.394917   868 exec.cpp:435] Executor asked to shutdown
I1212 18:41:56.394917  5784 executor.cpp:171] Received SHUTDOWN event
I1212 18:41:56.394917  5784 executor.cpp:747] Shutting down
I1212 18:41:56.394917  5784 executor.cpp:854] Sending SIGTERM to process tree 
at pid 5 status update state: TASK_KILLED)
I1212 18:41:56.392915  7048 slave.cpp:3400] Shutting down framework 
eb15fa9d-8a4c-4edb-a501-24251020b202-
I1212 18:41:56.392915  7048 slave.cpp:6091] Shutting down executor 
'd2e84fc2-29fe-4a26-8ad3-aae3fca54755' of framework 
eb15fa9d-8a4c-4edb-a501-24251020b202- at executor(1)@10.3.1.11:49611
I1212 18:41:56.393916  7048 slave.cpp:909] Agent terminating
W1212 18:41:56.393916  7048 slave.cpp:3396] Ignoring shutdown framework 
eb15fa9d-8a4c-4edb-a501-24251020b202- because it is terminating
I1212 18:41:56.394917  9544 master.cpp:10220] Removing task 
d2e84fc2-29fe-4a26-8ad3-aae3fca54755 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework eb15fa9d-8a4c-4edb-a501-24251020b202- on 
agent eb15fa9d-8a4c-4edb-a501-24251020b202-S0 at slave(326)@10.3.1.11:49589 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1212 18:41:56.397920  2768 containerizer.cpp:2328] Destroying container 
e3ade64e-2326-4d7b-8ea4-b121a0a7f633 in RUNNING state
I1212 18:41:56.397920  9544 master.cpp:1305] Agent 
eb15fa9d-8a4c-4edb-a501-24251020b202-S0 at slave(326)@10.3.1.11:49589 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1212 18:41:56.397920  9544 master.cpp:3364] Disconnecting agent 
eb15fa9d-8a4c-4edb-a501-24251020b202-S0 at slave(326)@10.3.1.11:49589

Re: Review Request 64505: Made the agent check for offer operation update retries.

2017-12-12 Thread Greg Mann 


> On Dec. 12, 2017, 1:08 a.m., Jie Yu wrote:
> > src/slave/slave.cpp
> > Lines 7296-7298 (patched)
> > 
> >
> > Wondering if we should do this check in `updateOfferOperation`?
> > 
> > What's the equality check for `OfferOperationStatus`? does the order in 
> > `Resources` matter?
> > 
> > Also, wondering what we do for Tasks in this situation?

Yea I considered putting this in `updateOfferOperation` instead. In the absence 
of other motivations, I opted for what seems like better readability: the 
function `updateOfferOperation` is used to update the agent's representation of 
an offer operation, and if we don't need to update, then we don't call it at 
all. LMK what you think.

Regarding equality, I can update the patch to just check the `status_uuid`.

Regarding tasks, I'll quote here from our offline discussion:
"in the task case, the `Slave::forward` function is only responsible for 
sending the update, not updating the agent’s internal task state - the agent’s 
state is updated in the `statusUpdate` code path (which also calls 
`taskStatusUpdateManager->update()`).
in the offer operation case, the LRP’s forwarding function submits an agent 
call, and then the agent’s handler is responsible for _both_ sending the update 
message and updating the agent’s internal state.

thus, in the task case we don’t have this issue - the `statusUpdate` code path 
is only executed once for each update, whereas for offer operations the code 
path which updates the agent’s state is executed every time we forward an 
update, including retries."


- Greg


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64505/#review193476
---


On Dec. 11, 2017, 8:17 p.m., Greg Mann wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64505/
> ---
> 
> (Updated Dec. 11, 2017, 8:17 p.m.)
> 
> 
> Review request for mesos, Benjamin Bannier, Jie Yu, and Jan Schlicht.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Local resource providers send all offer operation status updates
> using the reosurce provider API. This patch makes the agent's
> resource provider message handler skip operation updates when an
> update is a retry.
> 
> 
> Diffs
> -
> 
>   src/slave/slave.cpp 373e393ca1e7c0c30c3474cc9e630e25ad92f235 
> 
> 
> Diff: https://reviews.apache.org/r/64505/diff/1/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Greg Mann
> 
>

Re: Review Request 64548: Made the agent respond to reconciliation requests for default resources.

2017-12-12 Thread Jie Yu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64548/#review193582
---


Ship it!




Ship It!

- Jie Yu


On Dec. 12, 2017, 6:21 p.m., Greg Mann wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64548/
> ---
> 
> (Updated Dec. 12, 2017, 6:21 p.m.)
> 
> 
> Review request for mesos, Benjamin Bannier, Gaston Kleiman, Jie Yu, and Jan 
> Schlicht.
> 
> 
> Bugs: MESOS-8195
> https://issues.apache.org/jira/browse/MESOS-8195
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Made the agent respond to reconciliation requests for default resources.
> 
> 
> Diffs
> -
> 
>   src/slave/slave.cpp 5869e73ca1c14c99e580da9d7375181da2073ec5 
> 
> 
> Diff: https://reviews.apache.org/r/64548/diff/1/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Greg Mann
> 
>

Re: Review Request 64550: Made `update` return a future completed before `forwardCallback`.

2017-12-12 Thread Jie Yu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64550/#review193581
---


Ship it!




Ship It!

- Jie Yu


On Dec. 12, 2017, 5:49 p.m., Chun-Hung Hsiao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64550/
> ---
> 
> (Updated Dec. 12, 2017, 5:49 p.m.)
> 
> 
> Review request for mesos, Gaston Kleiman, Greg Mann, and Jie Yu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> We maintain an ordering invariant that the completion of the future
> returned by `update` happens before the invocation of `forwardCallback`.
> This invariant provides the guarantee that if `forwardCallback` is
> deferred to the same context as the chained callback of the returned
> future, the latter will happen before the earlier. This is useful if
> the caller of the status update manager want to maintain its internal
> state after the manager checkpoints the status update but before the
> status update is sent out.
> 
> 
> Diffs
> -
> 
>   src/status_update_manager/status_update_manager_process.hpp 
> 1ac64410594134ce3e709d396d0a74a3d0aa2007 
> 
> 
> Diff: https://reviews.apache.org/r/64550/diff/1/
> 
> 
> Testing
> ---
> 
> sudo make check
> 
> 
> Thanks,
> 
> Chun-Hung Hsiao
> 
>

Re: Review Request 64548: Made the agent respond to reconciliation requests for default resources.

2017-12-12 Thread Greg Mann 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64548/
---

(Updated Dec. 12, 2017, 6:21 p.m.)


Review request for mesos, Benjamin Bannier, Gaston Kleiman, Jie Yu, and Jan 
Schlicht.


Bugs: MESOS-8195
https://issues.apache.org/jira/browse/MESOS-8195


Repository: mesos


Description
---

Made the agent respond to reconciliation requests for default resources.


Diffs
-

  src/slave/slave.cpp 5869e73ca1c14c99e580da9d7375181da2073ec5 


Diff: https://reviews.apache.org/r/64548/diff/1/


Testing
---

make check


Thanks,

Greg Mann

Review Request 64548: Made the agent respond to reconciliation requests for default resources.

2017-12-12 Thread Greg Mann 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64548/
---

Review request for mesos, Benjamin Bannier, Gaston Kleiman, Jie Yu, and Jan 
Schlicht.


Repository: mesos


Description
---

Made the agent respond to reconciliation requests for default resources.


Diffs
-

  src/slave/slave.cpp 5869e73ca1c14c99e580da9d7375181da2073ec5 


Diff: https://reviews.apache.org/r/64548/diff/1/


Testing
---

make check


Thanks,

Greg Mann

Review Request 64551: Integrated the status update manager into SLRP.

2017-12-12 Thread Chun-Hung Hsiao 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64551/
---

Review request for mesos, Gaston Kleiman, Greg Mann, and Jie Yu.


Repository: mesos


Description
---

This patch uses the status update manager to send status updates and
recover operations that are checkpointed as completed in the status
update manager but still in the pending list in SLRP. It also forwards
the acknowledgements to the status update manager, and garbage collect
the metadata for offer operations.


Diffs
-

  src/resource_provider/daemon.hpp 7c513a2b7c7ca958f9ea57326493f74df5b67c8f 
  src/resource_provider/daemon.cpp 7c783e309672a2d93532fb17f13afcb96d5a7006 
  src/resource_provider/local.hpp 46111d9f8969ea47e695d08832cb5a9ea60d1971 
  src/resource_provider/local.cpp d1d683534d186e77a2beb32c978f2aaea289b086 
  src/resource_provider/storage/provider.hpp 
374f837ad693820afffb86bd1e54915f6796d088 
  src/resource_provider/storage/provider.cpp 
e806f44ef33405d4a2b133576c60be56e9fe3435 
  src/slave/paths.hpp d645d871c36bbe8e766a98650f6aa23b6eab65d8 
  src/slave/paths.cpp b8004e76964abc210820368a89dbfa6928ef7bfd 


Diff: https://reviews.apache.org/r/64551/diff/1/


Testing
---

sudo make check


Thanks,

Chun-Hung Hsiao

Re: Review Request 64475: Initialized offer operation status update manager in SLRP.

2017-12-12 Thread Chun-Hung Hsiao 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64475/
---

(Updated Dec. 12, 2017, 5:51 p.m.)


Review request for mesos, Gaston Kleiman, Greg Mann, and Jie Yu.


Changes
---

Addressed Greg's comments and moved the SUM initialization after SUBSCRIBED.


Repository: mesos


Description
---

This patch adds an agent filesystem layout for checkpointing offer
operation status updates for resource providers, and initialized
a status update manager in storage local resource provider.


Diffs (updated)
-

  src/resource_provider/storage/provider.cpp 
e806f44ef33405d4a2b133576c60be56e9fe3435 
  src/slave/paths.hpp d645d871c36bbe8e766a98650f6aa23b6eab65d8 
  src/slave/paths.cpp b8004e76964abc210820368a89dbfa6928ef7bfd 


Diff: https://reviews.apache.org/r/64475/diff/2/

Changes: https://reviews.apache.org/r/64475/diff/1-2/


Testing (updated)
---

sudo make check


Thanks,

Chun-Hung Hsiao

Review Request 64550: Made `update` return a future completed before `forwardCallback`.

2017-12-12 Thread Chun-Hung Hsiao 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64550/
---

Review request for mesos, Gaston Kleiman, Greg Mann, and Jie Yu.


Repository: mesos


Description
---

We maintain an ordering invariant that the completion of the future
returned by `update` happens before the invocation of `forwardCallback`.
This invariant provides the guarantee that if `forwardCallback` is
deferred to the same context as the chained callback of the returned
future, the latter will happen before the earlier. This is useful if
the caller of the status update manager want to maintain its internal
state after the manager checkpoints the status update but before the
status update is sent out.


Diffs
-

  src/status_update_manager/status_update_manager_process.hpp 
1ac64410594134ce3e709d396d0a74a3d0aa2007 


Diff: https://reviews.apache.org/r/64550/diff/1/


Testing
---

sudo make check


Thanks,

Chun-Hung Hsiao

Re: Review Request 64546: Fixed a flaky test.

2017-12-12 Thread Mesos Reviewbot Windows 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64546/#review193561
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64546']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64546

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64546/logs/mesos-tests-stdout.log):

```

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2311 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2332 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2253 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2276 ms total)

[--] Global test environment tear-down
[==] 829 tests from 84 test cases ran. (304197 ms total)
[  PASSED  ] 819 tests.
[  FAILED  ] 10 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] SlaveTest.ResourceProviderPublishAll

10 FAILED TESTS
  YOU HAVE 204 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64546/logs/mesos-tests-stderr.log):

```
I1212 16:37:27.830670  3076 executor.cpp:171] Received SUBSCRIBED event
I1212 16:37:27.834671  3076 executor.cpp:175] Subscribed executor on 
build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1212 16:37:27.834671  3076 executor.cpp:171] Received LAUNCH event
I1212 16:37:27.838670  3076 executor.cpp:637] Starting task 
2768344d-1ac0-45e6-8bc2-3905daa560f4
I1212 16:37:27.911669  3076 executor.cpp:477] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1212 16:37:28.450675  3076 executor.cpp:650] Forked command at 5056
I1212 16:37:28.476675  8596 exec.cpp:435] Executor asked to shutdown
I1212 16:37:28.477675  4156 executor.cpp:171] Received SHUTDOWN event
I1212 16:37:28.477675  4156 executor.cpp:747] Shutting down
I1212 16:37:28.477675  4156 executor.cpp:854] Sending SIGTERM to process tree 
at pid 54e43-af0a-2d48f2bae027@10.3.1.5:62719
I1212 16:37:28.474678  8848 hierarchical.cpp:405] Deactivated framework 
62af06ef-1d7c-4889-9d83-14175ba00a76-
I1212 16:37:28.475675  1608 master.cpp:10114] Updating the state of task 
2768344d-1ac0-45e6-8bc2-3905daa560f4 of framework 
62af06ef-1d7c-4889-9d83-14175ba00a76- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I1212 16:37:28.475675   992 slave.cpp:3400] Shutting down framework 
62af06ef-1d7c-4889-9d83-14175ba00a76-
I1212 16:37:28.475675   992 slave.cpp:6091] Shutting down executor 
'2768344d-1ac0-45e6-8bc2-3905daa560f4' of framework 
62af06ef-1d7c-4889-9d83-14175ba00a76- at executor(1)@10.3.1.5:62740
I1212 16:37:28.475675   992 slave.cpp:909] Agent terminating
W1212 16:37:28.476675   992 slave.cpp:3396] Ignoring shutdown framework 
62af06ef-1d7c-4889-9d83-14175ba00a76- because it is terminating
I1212 16:37:28.477675  1608 master.cpp:10220] Removing task 
2768344d-1ac0-45e6-8bc2-3905daa560f4 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework 62af06ef-1d7c-4889-9d83-14175ba00a76- on 
agent 62af06ef-1d7c-4889-9d83-14175ba00a76-S0 at slave(326)@10.3.1.5:62719 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1212 16:37:28.478704   992 containerizer.cpp:2328] Destroying container 
d662026c-6e79-4319-b016-f1c836a630a1 in RUNNING state
I1212 16:37:28.479676   992 containerizer.cpp:2930] Transitioning the state of 
container d662026c-6e79-4319-b016-f1c836a630a1 from RUNNING to DESTROYING
I1212 16:37:28.479676   992 launcher.cpp:156] Asked to destroy container 
d662026c-6e79-4319-b016-f1c836a630a1
I1212 16:37:28.480675  1608 master.cpp:1305] Agent 
62af06ef-1d7c-4889-9d83-14175ba00a76-S0 at slave(326)@10.3.1.5:62719 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1212

Re: Review Request 64098: Send status updates when agent re-registers.

2017-12-12 Thread Jiang Yan Xu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64098/#review193557
---


Ship it!




Ship It!

- Jiang Yan Xu


On Dec. 12, 2017, 4:54 a.m., Megha Sharma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64098/
> ---
> 
> (Updated Dec. 12, 2017, 4:54 a.m.)
> 
> 
> Review request for mesos, Ilya Pronin, James Peach, and Jiang Yan Xu.
> 
> 
> Bugs: MESOS-6406
> https://issues.apache.org/jira/browse/MESOS-6406
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Master will send task status updates to frameworks when an agent
> which has been previously removed by the master for being unreachable
> or is unknown to the master due to the garbage collection of
> the unreachable and gone agents in the registry re-registers.
> 
> 
> Diffs
> -
> 
>   src/master/master.cpp b10d0341276090bfa70aaa4fd6317a560e3334ea 
>   src/tests/master_allocator_tests.cpp 
> 10de6f0104a6a93ff5037d7e2ab5cf21d57fbec8 
>   src/tests/master_tests.cpp 7b087675200fe0ca69319b715997700a91d9f6b2 
>   src/tests/partition_tests.cpp 54ccf783682e7e6db0847b9a6313489b7f8181f8 
>   src/tests/persistent_volume_tests.cpp 
> f255382af957cfa66f5efaffcaf1082b83b35e58 
>   src/tests/slave_recovery_tests.cpp 253b0fc2ff7ec1f00937d42636151553c46d5175 
>   src/tests/upgrade_tests.cpp 0efaa586153564b20f7884023946a11635425ee4 
> 
> 
> Diff: https://reviews.apache.org/r/64098/diff/15/
> 
> 
> Testing
> ---
> 
> with make check
> 
> 
> Thanks,
> 
> Megha Sharma
> 
>

Re: Review Request 64543: Exposed resource provider information in master 'GET_AGENTS' response.

2017-12-12 Thread Mesos Reviewbot Windows 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64543/#review193545
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64543']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64543

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64543/logs/mesos-tests-stdout.log):

```

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2369 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2396 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2480 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2505 ms total)

[--] Global test environment tear-down
[==] 829 tests from 84 test cases ran. (319985 ms total)
[  PASSED  ] 819 tests.
[  FAILED  ] 10 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] SlaveTest.ResourceProviderPublishAll

10 FAILED TESTS
  YOU HAVE 201 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64543/logs/mesos-tests-stderr.log):

```
I1212 15:40:19.733963  7092 master.cpp:10114] Updating the state of task 
031455f5-28e6-4b0c-8ece-02f36d5d548d of framework 
28b78b9a-209d-4a82-a559-7f9671fef77e- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I1212 15:40:19.733963   448 slave.cpp:3400] Shutting down framework 
28b78b9a-209d-4a82-a559-7f9671fef77e-
I1212 15:40:19.733963   448 slave.cpp:6091] Shutting down executor 
'031455f5-28e6-4b0c-8ece-02f36d5d548d' of framework 
28b78b9a-209d-4a82-a559-7f9671fef77e- at executor(1)@10.3.1.11:63951
I1212 15:40:19.734962   448 slave.cpp:909] Agent terminating
W1212 15:40:19.734962   448 slave.cpp:3396] Ignoring shutdown framework 
28b78b9a-209d-4a82-a559-7f9671fef77e- because it is terminating
I1212 15:40:19.735962  7092 master.cpp:10220] Removing taskI1212 
15:40:19.018965  9104 exec.cpp:162] Version: 1.5.0
I1212 15:40:19.042210  8012 exec.cpp:237] Executor registered on agent 
28b78b9a-209d-4a82-a559-7f9671fef77e-S0
I1212 15:40:19.043965  1108 executor.cpp:171] Received SUBSCRIBED event
I1212 15:40:19.048961  1108 executor.cpp:175] Subscribed executor on 
build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1212 15:40:19.049960  1108 executor.cpp:171] Received LAUNCH event
I1212 15:40:19.053961  1108 executor.cpp:637] Starting task 
031455f5-28e6-4b0c-8ece-02f36d5d548d
I1212 15:40:19.129961  1108 executor.cpp:477] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1212 15:40:19.702962  1108 executor.cpp:650] Forked command at 3744
I1212 15:40:19.735962  9252 exec.cpp:435] Executor asked to shutdown
I1212 15:40:19.735962  1108 executor.cpp:171] Received SHUTDOWN event
I1212 15:40:19.735962  1108 executor.cpp:747] Shutting down
I1212 15:40:19.735962  1108 executor.cpp:854] Sending SIGTERM to process tree 
at pid 3 031455f5-28e6-4b0c-8ece-02f36d5d548d with resources cpus(allocated: 
*):4; mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework 28b78b9a-209d-4a82-a559-7f9671fef77e- on 
agent 28b78b9a-209d-4a82-a559-7f9671fef77e-S0 at slave(326)@10.3.1.11:63930 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1212 15:40:19.737962  8644 containerizer.cpp:2328] Destroying container 
aa3f40cf-910d-4bde-bd14-c53c8eb38d1a in RUNNING state
I1212 15:40:19.737962  8644 containerizer.cpp:2930] Transitioning the state of 
container aa3f40cf-910d-4bde-bd14-c53c8eb38d1a from RUNNING to DESTROYING
I1212 15:40:19.738965  7092 master.cpp:1305] Agent 
28b78b9a-209d-4a82-a559-7f9671fef77e-S0 at slave(326)@10.3.1.11:63930 
(build-srv-03.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1212 15:40:19.738965  7092 master.cpp:3364] Disconnecting agent 
28b78b9a-209d-4a82-a559-7f9671fef77e-S0

Re: Review Request 64098: Send status updates when agent re-registers.

2017-12-12 Thread Mesos Reviewbot Windows 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64098/#review193544
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64250', '64098']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64098

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64098/logs/mesos-tests-stdout.log):

```

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2284 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2306 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2265 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2287 ms total)

[--] Global test environment tear-down
[==] 829 tests from 84 test cases ran. (305398 ms total)
[  PASSED  ] 819 tests.
[  FAILED  ] 10 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] SlaveTest.ResourceProviderPublishAll

10 FAILED TESTS
  YOU HAVE 204 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64098/logs/mesos-tests-stderr.log):

```
I1212 15:36:49.166775  6504 master.cpp:10149] Updating the state of task 
12db25fd-0b6f-4f66-9b84-e1a1c13e46ff of framework 
d72ff96e-44f0-4c60-bf3f-16dff914edd0- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I1212 15:36:49.1I1212 15:36:48.500752  7276 exec.cpp:162] Version: 1.5.0
I1212 15:36:48.523741   508 exec.cpp:237] Executor registered on agent 
d72ff96e-44f0-4c60-bf3f-16dff914edd0-S0
I1212 15:36:48.525737  4216 executor.cpp:171] Received SUBSCRIBED event
I1212 15:36:48.529757  4216 executor.cpp:175] Subscribed executor on 
build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1212 15:36:48.530753  4216 executor.cpp:171] Received LAUNCH event
I1212 15:36:48.533756  4216 executor.cpp:637] Starting task 
12db25fd-0b6f-4f66-9b84-e1a1c13e46ff
I1212 15:36:48.606755  4216 executor.cpp:477] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1212 15:36:49.136749  4216 executor.cpp:650] Forked command at 6088
I1212 15:36:49.168748  7684 exec.cpp:435] Executor asked to shutdown
I1212 15:36:49.169749  8940 executor.cpp:171] Received SHUTDOWN event
I1212 15:36:49.169749  8940 executor.cpp:747] Shutting down
I1212 15:36:49.169749  8940 executor.cpp:854] Sending SIGTERM to process tree 
at pid 666775  5788 slave.cpp:3400] Shutting down framework 
d72ff96e-44f0-4c60-bf3f-16dff914edd0-
I1212 15:36:49.167755  5788 slave.cpp:6091] Shutting down executor 
'12db25fd-0b6f-4f66-9b84-e1a1c13e46ff' of framework 
d72ff96e-44f0-4c60-bf3f-16dff914edd0- at executor(1)@10.3.1.5:60853
I1212 15:36:49.168748  6504 master.cpp:10255] Removing task 
12db25fd-0b6f-4f66-9b84-e1a1c13e46ff with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework d72ff96e-44f0-4c60-bf3f-16dff914edd0- on 
agent d72ff96e-44f0-4c60-bf3f-16dff914edd0-S0 at slave(326)@10.3.1.5:60832 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1212 15:36:49.169749  5532 slave.cpp:909] Agent terminating
W1212 15:36:49.169749  5532 slave.cpp:3396] Ignoring shutdown framework 
d72ff96e-44f0-4c60-bf3f-16dff914edd0- because it is terminating
I1212 15:36:49.170748  6504 master.cpp:1305] Agent 
d72ff96e-44f0-4c60-bf3f-16dff914edd0-S0 at slave(326)@10.3.1.5:60832 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1212 15:36:49.171747  6504 master.cpp:3364] Disconnecting agent 
d72ff96e-44f0-4c60-bf3f-16dff914edd0-S0 at slave(326)@10.3.1.5:60832 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1212 15:36:49.171747  6504 master.cpp:3383] Deactivating agent 
d72ff96e-44f0-4c60-bf3f-16dff914edd0-S0 at slave(326)@10.3.1.5:60832

Re: Review Request 64543: Exposed resource provider information in master 'GET_AGENTS' response.

2017-12-12 Thread Jan Schlicht 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64543/#review193543
---


Ship it!




Ship It!

- Jan Schlicht


On Dec. 12, 2017, 3:38 p.m., Benjamin Bannier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64543/
> ---
> 
> (Updated Dec. 12, 2017, 3:38 p.m.)
> 
> 
> Review request for mesos, Jie Yu and Jan Schlicht.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This patch extends the master's 'GET_AGENTS' response to include local
> resource provider information. For now only the 'ResourceProviderInfo'
> are shown.
> 
> 
> Diffs
> -
> 
>   include/mesos/master/master.proto 87083beec66a9f57803bfb0735188fcdedafaa87 
>   include/mesos/v1/master/master.proto 
> ea04b8f1e28e1c20ac803f8f1610cc1e765ddd9d 
>   src/master/http.cpp bb040ced65495cfe78b67ca7258499c16ac5d8a1 
>   src/tests/api_tests.cpp 3372849cb2aa070df6b2262b57baa0fab628e11d 
> 
> 
> Diff: https://reviews.apache.org/r/64543/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Benjamin Bannier
> 
>

Re: Review Request 64032: Promoted log level to warning for disconnected events in exec.cpp.

2017-12-12 Thread Andrei Budnik 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64032/#review193542
---


Ship it!




Ship It!

- Andrei Budnik


On Dec. 4, 2017, 6:03 p.m., Alexander Rukletsov wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64032/
> ---
> 
> (Updated Dec. 4, 2017, 6:03 p.m.)
> 
> 
> Review request for mesos, Andrei Budnik, Anand Mazumdar, Armand Grillet, and 
> Vinod Kone.
> 
> 
> Bugs: MESOS-8297
> https://issues.apache.org/jira/browse/MESOS-8297
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> When the executor library receives messages while being disconnected,
> it might indicate an out-of-order message delivery or lost messages.
> This should be logged at the warning level to simplify triaging.
> 
> 
> Diffs
> -
> 
>   src/exec/exec.cpp cdbf149a9047ddad6beef64be1266e15e7643afc 
> 
> 
> Diff: https://reviews.apache.org/r/64032/diff/2/
> 
> 
> Testing
> ---
> 
> None: not a functional change.
> 
> 
> Thanks,
> 
> Alexander Rukletsov
> 
>

Review Request 64546: Fixed a flaky test.

2017-12-12 Thread Jan Schlicht 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64546/
---

Review request for mesos, Alexander Rukletsov, Benjamin Bannier, and Jie Yu.


Bugs: MESOS-8315
https://issues.apache.org/jira/browse/MESOS-8315


Repository: mesos


Description
---

Callbacks of already invalidated instances could be called in
'ResubscribeResourceProvider' when simulating a resource provider
restart. This has been resolved by not only restarting a resource
provider's driver, but creating a new 'MockResourceProvider' instance.


Diffs
-

  src/tests/resource_provider_manager_tests.cpp 
e37a53ac6a03e2ea58dd6580fc8a399a1398d950 


Diff: https://reviews.apache.org/r/64546/diff/1/


Testing
---

make check

src/mesos-tests 
--gtest_filter=*ResourceProviderManagerHttpApiTest.ResubscribeResourceProvider* 
--gtest_repeat=1000 --gtest_break_on_failure (with 'stress' running in the 
background)


Thanks,

Jan Schlicht

Re: Review Request 64168: Introduced a 'UUID' type.

2017-12-12 Thread Mesos Reviewbot Windows 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64168/#review193541
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64381', '64380', '64168']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64168

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64168/logs/mesos-tests-stdout.log):

```

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2313 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2347 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2252 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2274 ms total)

[--] Global test environment tear-down
[==] 829 tests from 84 test cases ran. (303365 ms total)
[  PASSED  ] 819 tests.
[  FAILED  ] 10 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] SlaveTest.ResourceProviderPublishAll

10 FAILED TESTS
  YOU HAVE 204 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64168/logs/mesos-tests-stderr.log):

```
I1212 14:47:47.116789  3380 master.cpp:10119] Updating the state of task 
67b5decf-d502-4f58-a8e6-7455963fffc6 of framework 
f3cba6e6-efb2-41f5-8ff7-e113306e5246- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I1212 14:47:47.116789  6264 slave.cpp:3401] Shutting down framework 
f3cba6e6-efb2-41f5-8ff7-e113306e5246-
I1212 14:47:47.117807  6264 slave.cpp:6093] Shutting down executor 
'67b5decf-d502-4f58-a8e6-7455963fffc6' of framework 
f3cba6e6-efb2-41f5-8ff7-e113306e5246- at executor(1)@10.3.1.5:58993
I1212 14:47:47.117807  6264 slave.cpp:909] Agent terminating
W1212 14:47:47.118788  6264 slave.cpp:3397I1212 14:47:46.458775  1940 
exec.cpp:162] Version: 1.5.0
I1212 14:47:46.481771  7744 exec.cpp:237] Executor registered on agent 
f3cba6e6-efb2-41f5-8ff7-e113306e5246-S0
I1212 14:47:46.484752   940 executor.cpp:171] Received SUBSCRIBED event
I1212 14:47:46.488776   940 executor.cpp:175] Subscribed executor on 
build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1212 14:47:46.488776   940 executor.cpp:171] Received LAUNCH event
I1212 14:47:46.492776   940 executor.cpp:638] Starting task 
67b5decf-d502-4f58-a8e6-7455963fffc6
I1212 14:47:46.566776   940 executor.cpp:478] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1212 14:47:47.093786   940 executor.cpp:651] Forked command at 3228
I1212 14:47:47.118788  8020 exec.cpp:435] Executor asked to shutdown
I1212 14:47:47.118788  2636 executor.cpp:171] Received SHUTDOWN event
I1212 14:47:47.119788  2636 executor.cpp:748] Shutting down
I1212 14:47:47.119788  2636 executor.cpp:855] Sending SIGTERM to process tree 
at pid 3] Ignoring shutdown framework f3cba6e6-efb2-41f5-8ff7-e113306e5246- 
because it is terminating
I1212 14:47:47.119788  3380 master.cpp:10225] Removing task 
67b5decf-d502-4f58-a8e6-7455963fffc6 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework f3cba6e6-efb2-41f5-8ff7-e113306e5246- on 
agent f3cba6e6-efb2-41f5-8ff7-e113306e5246-S0 at slave(326)@10.3.1.5:58972 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1212 14:47:47.121788  3144 containerizer.cpp:2328] Destroying container 
f2fdea5d-fde8-4d70-a6b6-8cf9ffdd282b in RUNNING state
I1212 14:47:47.121788  3144 containerizer.cpp:2930] Transitioning the state of 
container f2fdea5d-fde8-4d70-a6b6-8cf9ffdd282b from RUNNING to DESTROYING
I1212 14:47:47.121788  3380 master.cpp:1305] Agent 
f3cba6e6-efb2-41f5-8ff7-e113306e5246-S0 at slave(326)@10.3.1.5:58972 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1212 14:47:47.121788  3380 master.cpp:3364] Disconnecting agent

Review Request 64543: Exposed resource provider information in master 'GET_AGENTS' response.

2017-12-12 Thread Benjamin Bannier 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64543/
---

Review request for mesos, Jie Yu and Jan Schlicht.


Repository: mesos


Description
---

This patch extends the master's 'GET_AGENTS' response to include local
resource provider information. For now only the 'ResourceProviderInfo'
are shown.


Diffs
-

  include/mesos/master/master.proto 87083beec66a9f57803bfb0735188fcdedafaa87 
  include/mesos/v1/master/master.proto ea04b8f1e28e1c20ac803f8f1610cc1e765ddd9d 
  src/master/http.cpp bb040ced65495cfe78b67ca7258499c16ac5d8a1 
  src/tests/api_tests.cpp 3372849cb2aa070df6b2262b57baa0fab628e11d 


Diff: https://reviews.apache.org/r/64543/diff/1/


Testing
---


Thanks,

Benjamin Bannier

Re: Review Request 64168: Introduced a 'UUID' type.

2017-12-12 Thread Jan Schlicht 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64168/
---

(Updated Dec. 12, 2017, 3:20 p.m.)


Review request for mesos, Benjamin Bannier and Jie Yu.


Changes
---

Addressed an issue.


Bugs: MESOS-8309
https://issues.apache.org/jira/browse/MESOS-8309


Repository: mesos


Description
---

Introduced a 'UUID' type.


Diffs (updated)
-

  include/mesos/mesos.proto 0ce25981f9fc7a45c76e8f0b0e75fb9836b9fab0 
  include/mesos/resource_provider/resource_provider.proto 
7ec693628451abd3c4da800a021a488c2d01eb50 
  include/mesos/type_utils.hpp 1bcc521546cf017ca5b33cb4edd509ce3aa64d5a 
  include/mesos/v1/mesos.proto 10ce9493a86707b462268df4dca77c0a6a3b78d8 
  include/mesos/v1/resource_provider/resource_provider.proto 
31e8954b053c1541d2d0eba90ae9d0a7051b74ce 
  src/common/protobuf_utils.cpp c5504a00a7a84b824743f00c6b97ea299ac66eb4 
  src/common/type_utils.cpp 65586a57b6d384350673903fb399b89a83fb9c37 
  src/master/master.hpp 232cc3758f240db626c4fdaf852163fa48af4dd7 
  src/master/master.cpp b10d0341276090bfa70aaa4fd6317a560e3334ea 
  src/messages/messages.cpp 56876f2d5a92237ccb85ec6d8ab8eb872061a7b5 
  src/messages/messages.proto e680cd5e4d5a93c3c77309f327844f55fbb239a1 
  src/resource_provider/manager.cpp fd138b9914d925b5be7a11255dd632921c107dba 
  src/resource_provider/message.hpp eab90cffd6aab9e38207dcf109cc737171ed3953 
  src/slave/slave.cpp d2be4f4001cd30a927bb9d8675997185609ade1e 
  src/status_update_manager/offer_operation.cpp 
f66690eadc0d7b16cc7de1b518904db2cab7ef82 
  src/status_update_manager/status_update_manager_process.hpp 
1ac64410594134ce3e709d396d0a74a3d0aa2007 
  src/tests/mesos.hpp 320e9755c73bcf19ba8ac87f2081f17fdaa438ea 
  src/tests/offer_operation_status_update_manager_tests.cpp 
a5327d3d070a390f2ceb918dc53a024fbfe8777e 
  src/tests/resource_provider_manager_tests.cpp 
e37a53ac6a03e2ea58dd6580fc8a399a1398d950 
  src/tests/resource_provider_validation_tests.cpp 
bf789a05771b7c25f2fc2a8a5b35d38519e4793b 
  src/tests/slave_tests.cpp 5228e03af1585817108094c587da6ae30d67df67 


Diff: https://reviews.apache.org/r/64168/diff/6/

Changes: https://reviews.apache.org/r/64168/diff/5-6/


Testing
---

make check


Thanks,

Jan Schlicht

Re: Review Request 64168: Introduced a 'UUID' type.

2017-12-12 Thread Jan Schlicht 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64168/
---

(Updated Dec. 12, 2017, 2:59 p.m.)


Review request for mesos, Benjamin Bannier and Jie Yu.


Changes
---

Rebased and addressed issues.


Bugs: MESOS-8309
https://issues.apache.org/jira/browse/MESOS-8309


Repository: mesos


Description
---

Introduced a 'UUID' type.


Diffs (updated)
-

  include/mesos/mesos.proto 0ce25981f9fc7a45c76e8f0b0e75fb9836b9fab0 
  include/mesos/resource_provider/resource_provider.proto 
7ec693628451abd3c4da800a021a488c2d01eb50 
  include/mesos/type_utils.hpp 1bcc521546cf017ca5b33cb4edd509ce3aa64d5a 
  include/mesos/v1/mesos.proto 10ce9493a86707b462268df4dca77c0a6a3b78d8 
  include/mesos/v1/resource_provider/resource_provider.proto 
31e8954b053c1541d2d0eba90ae9d0a7051b74ce 
  src/common/protobuf_utils.cpp c5504a00a7a84b824743f00c6b97ea299ac66eb4 
  src/common/type_utils.cpp 65586a57b6d384350673903fb399b89a83fb9c37 
  src/master/master.hpp 232cc3758f240db626c4fdaf852163fa48af4dd7 
  src/master/master.cpp b10d0341276090bfa70aaa4fd6317a560e3334ea 
  src/messages/messages.cpp 56876f2d5a92237ccb85ec6d8ab8eb872061a7b5 
  src/messages/messages.proto e680cd5e4d5a93c3c77309f327844f55fbb239a1 
  src/resource_provider/manager.cpp fd138b9914d925b5be7a11255dd632921c107dba 
  src/resource_provider/message.hpp eab90cffd6aab9e38207dcf109cc737171ed3953 
  src/slave/slave.cpp d2be4f4001cd30a927bb9d8675997185609ade1e 
  src/status_update_manager/offer_operation.cpp 
f66690eadc0d7b16cc7de1b518904db2cab7ef82 
  src/status_update_manager/status_update_manager_process.hpp 
1ac64410594134ce3e709d396d0a74a3d0aa2007 
  src/tests/mesos.hpp 320e9755c73bcf19ba8ac87f2081f17fdaa438ea 
  src/tests/offer_operation_status_update_manager_tests.cpp 
a5327d3d070a390f2ceb918dc53a024fbfe8777e 
  src/tests/resource_provider_manager_tests.cpp 
e37a53ac6a03e2ea58dd6580fc8a399a1398d950 
  src/tests/resource_provider_validation_tests.cpp 
bf789a05771b7c25f2fc2a8a5b35d38519e4793b 
  src/tests/slave_tests.cpp 5228e03af1585817108094c587da6ae30d67df67 


Diff: https://reviews.apache.org/r/64168/diff/5/

Changes: https://reviews.apache.org/r/64168/diff/4-5/


Testing
---

make check


Thanks,

Jan Schlicht

Re: Review Request 64381: Mesos: Used a namespace for UUID.

2017-12-12 Thread Jan Schlicht 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64381/
---

(Updated Dec. 12, 2017, 2:57 p.m.)


Review request for mesos, Benjamin Bannier, Jie Yu, and Vinod Kone.


Changes
---

Rebased and changed order.


Bugs: MESOS-8309
https://issues.apache.org/jira/browse/MESOS-8309


Repository: mesos


Description
---

To avoid conflicts when introducing a UUID type in Mesos' namespace,
Stout's UUID type is put in the 'id' namespace.


Diffs (updated)
-

  include/mesos/state/in_memory.hpp 203274242854d482b01275597b249c58e6dfe2ad 
  include/mesos/state/leveldb.hpp 6c732d38d68a3d60d28ce68a6340e8771d849c53 
  include/mesos/state/log.hpp ac0312fdb92c46bfa2a7b83e95e04fd1eaf87d03 
  include/mesos/state/state.hpp 8f7f3ad0bda029e03d9bdb874cb26add3f888522 
  include/mesos/state/storage.hpp 2bfa0478b0edf76d592cc9644da83d15a00bc68c 
  include/mesos/state/zookeeper.hpp 8d8c19ce778f2499d86eb84008a61f211c528a3a 
  src/checks/checker_process.cpp 496bd41811a1984e905c851b5e990495517c17b8 
  src/common/protobuf_utils.hpp 03b7cafb264b07a5a27dbd8d5c074e96ce576ea0 
  src/common/protobuf_utils.cpp c5504a00a7a84b824743f00c6b97ea299ac66eb4 
  src/examples/long_lived_executor.cpp 319db93eff8fcd347b03a17f78f5e9b6ea01e7d3 
  src/examples/persistent_volume_framework.cpp 
9ccc7ebe3079823042a8e93313d378b7f892249b 
  src/examples/test_http_executor.cpp 2166faf576980b58a00c53f9a889bfaeddefdd29 
  src/exec/exec.cpp cdbf149a9047ddad6beef64be1266e15e7643afc 
  src/executor/executor.cpp 7280951909b677e2a7b0afbcddba4a299dbe6f06 
  src/launcher/default_executor.cpp 248c46dbda3f63d8087f75120866c72e43aa4ed6 
  src/launcher/executor.cpp c688c04e598ac140421fd5e47359b0e48d30bcc5 
  src/master/http.cpp bb040ced65495cfe78b67ca7258499c16ac5d8a1 
  src/master/master.hpp 232cc3758f240db626c4fdaf852163fa48af4dd7 
  src/master/master.cpp b10d0341276090bfa70aaa4fd6317a560e3334ea 
  src/master/validation.cpp a9b0805179e87542f9127abebc372d901418 
  src/messages/messages.cpp 56876f2d5a92237ccb85ec6d8ab8eb872061a7b5 
  src/resource_provider/daemon.cpp 7c783e309672a2d93532fb17f13afcb96d5a7006 
  src/resource_provider/http_connection.hpp 
47686a8025c34b61c9d6f69eb20a0917f0635776 
  src/resource_provider/manager.cpp fd138b9914d925b5be7a11255dd632921c107dba 
  src/resource_provider/message.hpp eab90cffd6aab9e38207dcf109cc737171ed3953 
  src/sched/sched.cpp 72bd4138c6706d09c9c88a580f560dbdedf94574 
  src/scheduler/scheduler.cpp e35470933f9f4d3c795d4630bf64db7fc8808ce9 
  src/slave/containerizer/docker.cpp 229891da1b23ca1a5cf2b6b15cd16de6fa0d8240 
  src/slave/containerizer/mesos/io/switchboard.cpp 
8a00e6d47fe3d5415ca9202d6f37441fb6f12be7 
  src/slave/containerizer/mesos/isolators/volume/secret.cpp 
ba23519f7326d8ac1397d5c521efcb726914f7ea 
  src/slave/containerizer/mesos/provisioner/provisioner.cpp 
a17774449155aea1997a717230990ff93f7261f7 
  src/slave/slave.hpp 7c40fc71b49057fea0cfd85290931fbd0f6a9d62 
  src/slave/slave.cpp d2be4f4001cd30a927bb9d8675997185609ade1e 
  src/slave/state.hpp 8d088d02b786e1d4bf0187f4a574a0222811d10c 
  src/slave/state.cpp 3396157d2a5917e6128030e84295f5862d544d98 
  src/slave/task_status_update_manager.hpp 
6bdb468908bd5d9763bccdbc1a5667a530527ea3 
  src/slave/task_status_update_manager.cpp 
1ec6be73043a81dcf00c522ccc80bb656af13026 
  src/slave/validation.cpp 17cabd1c1c741858116a9c55ad4ee5b529edd84c 
  src/state/in_memory.cpp 2539b4f57223514bcacd6c62bb01ef3a6f24aa73 
  src/state/leveldb.cpp 9ced03a4a9f59fd949fc0735b2efcc6711dcfd1a 
  src/state/log.cpp b71383906cf28fe0769cbb620387a0e0134f01f9 
  src/state/zookeeper.cpp 1abd81651d575558f18001de89e9abef3a80caf4 
  src/status_update_manager/offer_operation.hpp 
8751f0b7b6471cab2db55bf52f45c2798e3ef192 
  src/status_update_manager/offer_operation.cpp 
f66690eadc0d7b16cc7de1b518904db2cab7ef82 
  src/status_update_manager/status_update_manager_process.hpp 
1ac64410594134ce3e709d396d0a74a3d0aa2007 
  src/tests/agent_container_api_tests.cpp 
595ce510783b6cdf219eadd287c57bd576d6443b 
  src/tests/api_tests.cpp 3372849cb2aa070df6b2262b57baa0fab628e11d 
  src/tests/container_daemon_tests.cpp 6e5efc952edeab0a03b33dcff81fec484126d068 
  src/tests/containerizer/composing_containerizer_tests.cpp 
1eab05ce488cf94afc2f5a6c2308e5c8db6f28b6 
  src/tests/containerizer/docker_containerizer_tests.cpp 
629c7a87f2843453b594c57f74f88e5f0b7aac6f 
  src/tests/containerizer/io_switchboard_tests.cpp 
fe9643a8fea9963ac476e52697f03e5ac3ead2bf 
  src/tests/containerizer/isolator_tests.cpp 
b341a86bfbc36a2b7b723236cafddaec10ee0ee6 
  src/tests/containerizer/linux_filesystem_isolator_tests.cpp 
4d796a8d46d667593f368daacadddaa5a292c907 
  src/tests/containerizer/mesos_containerizer_tests.cpp 
98adcfcfba4e5ee975b7ed0c073758ec2726763d 
  src/tests/containerizer/nested_mesos_containerizer_tests.cpp 
efc57ada58476e8265fccfa1e1a0a152029e1bf6

Re: Review Request 64380: Stout: Used a namespace for UUID.

2017-12-12 Thread Jan Schlicht 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64380/
---

(Updated Dec. 12, 2017, 2:56 p.m.)


Review request for mesos, Benjamin Bannier, Jie Yu, and Vinod Kone.


Changes
---

Rebased and changed order of reviews.


Bugs: MESOS-8309
https://issues.apache.org/jira/browse/MESOS-8309


Repository: mesos


Description
---

To avoid conflicts when introducing a UUID type in Mesos' namespace,
Stout's UUID type is put in the 'id' namespace.


Diffs (updated)
-

  3rdparty/stout/include/stout/uuid.hpp 
16830840c3528c9c26f57393cfdb73a09558a369 
  3rdparty/stout/tests/os/filesystem_tests.cpp 
57e1dc110c9fcb4960f59f4845b8255f0499df40 
  3rdparty/stout/tests/os_tests.cpp 02f2a18b861f185a0dc80c62a69c4b8e15bc1ac2 
  3rdparty/stout/tests/protobuf_tests.cpp 
543f96cfe8d9b2f857ed31d323596dc25f9f2d44 
  3rdparty/stout/tests/uuid_tests.cpp 1fbd623cb9902a60eb80e90252ed59a09910ea3e 


Diff: https://reviews.apache.org/r/64380/diff/3/

Changes: https://reviews.apache.org/r/64380/diff/2-3/


Testing (updated)
---

make check


Thanks,

Jan Schlicht

Re: Review Request 64098: Send status updates when agent re-registers.

2017-12-12 Thread Mesos Reviewbot Windows 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64098/#review193536
---



FAIL: Failed to apply the current review.

Failed command: `python.exe .\support\apply-reviews.py -n -r 64098`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64098

Relevant logs:

- 
[apply-review-64098-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64098/logs/apply-review-64098-stdout.log):

```
error: patch failed: src/master/master.cpp:6804
error: src/master/master.cpp: patch does not apply
```

- Mesos Reviewbot Windows


On Dec. 12, 2017, 6:24 p.m., Megha Sharma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64098/
> ---
> 
> (Updated Dec. 12, 2017, 6:24 p.m.)
> 
> 
> Review request for mesos, Ilya Pronin, James Peach, and Jiang Yan Xu.
> 
> 
> Bugs: MESOS-6406
> https://issues.apache.org/jira/browse/MESOS-6406
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Master will send task status updates to frameworks when an agent
> which has been previously removed by the master for being unreachable
> or is unknown to the master due to the garbage collection of
> the unreachable and gone agents in the registry re-registers.
> 
> 
> Diffs
> -
> 
>   src/master/master.cpp dfe60ef670edcaefa0c1241df2e2870f650fcf9e 
>   src/tests/master_allocator_tests.cpp 
> 3400d70bb0ba564eac43c4639eee0efd4d8059e6 
>   src/tests/master_tests.cpp 57eae320a7a398527cd3623c89bf67f319a8e955 
>   src/tests/partition_tests.cpp 31ebfe1655438eceae74d72a223df03a9dbd282d 
>   src/tests/persistent_volume_tests.cpp 
> 4aa3c2e8b0f461cd78053707cff8bcb2e6f2b0d7 
>   src/tests/slave_recovery_tests.cpp f14c6ef69eb20a03454c8197df79b572a3c6d050 
>   src/tests/upgrade_tests.cpp 7f434dbba858f636719eec24e92b306b76430c4c 
> 
> 
> Diff: https://reviews.apache.org/r/64098/diff/14/
> 
> 
> Testing
> ---
> 
> with make check
> 
> 
> Thanks,
> 
> Megha Sharma
> 
>

Re: Review Request 64536: Removed race from SlaveRecoveryTest.ReconnectExecutor.

2017-12-12 Thread Mesos Reviewbot Windows 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64536/#review193535
---



FAIL: Some Mesos tests failed.

Reviews applied: `['64536']`

Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`

All the build artifacts available at: 
http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64536

Relevant logs:

- 
[mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64536/logs/mesos-tests-stdout.log):

```

[--] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN  ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[   OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2203 ms)
[--] 1 test from IsolationFlag/CpuIsolatorTest (2226 ms total)

[--] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN  ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[   OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2232 ms)
[--] 1 test from IsolationFlag/MemoryIsolatorTest (2255 ms total)

[--] Global test environment tear-down
[==] 829 tests from 84 test cases ran. (306069 ms total)
[  PASSED  ] 819 tests.
[  FAILED  ] 10 tests, listed below:
[  FAILED  ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[  FAILED  ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[  FAILED  ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[  FAILED  ] 
OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[  FAILED  ] SlaveTest.ResourceProviderPublishAll

10 FAILED TESTS
  YOU HAVE 204 DISABLED TESTS

```

- 
[mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64536/logs/mesos-tests-stderr.log):

```
I1212 12:57:17.451431  7052 slave.cpp:3400] Shutting down framework 
841bed6f-0c12-4c5f-b3af-74557bd90411-
I1212 12:57:17.451431  9148 master.cpp:10114] Updating the state of task 
6d538a57-09b1-4fa1-b70d-6175711b5a35 of framework 
841bed6f-0c12-4c5f-b3af-74557bd90411- (latest state: TASK_KILLED, status 
update state: TASK_KILLED)
I1212 12:57:17.451431  7052 slave.cpp:6091] Shutting down executor 
'6d538a57-09b1-4fa1-b70d-6175711b5a35' of framework 
841bed6f-0c12-4c5f-b3af-74557bd90411- at executor(1)@10.3.1.5:57016
I1212 12:57:17.452409  7052 slave.cpp:909] Agent terminatiI1212 12:57:16.787437 
 1416 exec.cpp:162] Version: 1.5.0
I1212 12:57:16.810417  6384 exec.cpp:237] Executor registered on agent 
841bed6f-0c12-4c5f-b3af-74557bd90411-S0
I1212 12:57:16.813416  8364 executor.cpp:171] Received SUBSCRIBED event
I1212 12:57:16.817443  8364 executor.cpp:175] Subscribed executor on 
build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1212 12:57:16.817443  8364 executor.cpp:171] Received LAUNCH event
I1212 12:57:16.821440  8364 executor.cpp:637] Starting task 
6d538a57-09b1-4fa1-b70d-6175711b5a35
I1212 12:57:16.908439  8364 executor.cpp:477] Running 
'D:\DCOS\mesos\src\mesos-containerizer.exe launch '
I1212 12:57:17.428408  8364 executor.cpp:650] Forked command at 144
I1212 12:57:17.453408  1456 exec.cpp:435] Executor asked to shutdown
I1212 12:57:17.453408  4956 executor.cpp:171] Received SHUTDOWN event
I1212 12:57:17.453408  4956 executor.cpp:747] Shutting down
I1212 12:57:17.454408  4956 executor.cpp:854] Sending SIGTERM to process tree 
at pid 14ng
W1212 12:57:17.453408  7052 slave.cpp:3396] Ignoring shutdown framework 
841bed6f-0c12-4c5f-b3af-74557bd90411- because it is terminating
I1212 12:57:17.454408  9148 master.cpp:10220] Removing task 
6d538a57-09b1-4fa1-b70d-6175711b5a35 with resources cpus(allocated: *):4; 
mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: 
*):[31000-32000] of framework 841bed6f-0c12-4c5f-b3af-74557bd90411- on 
agent 841bed6f-0c12-4c5f-b3af-74557bd90411-S0 at slave(326)@10.3.1.5:56995 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1212 12:57:17.456408  7052 containerizer.cpp:2328] Destroying container 
42def7e4-36d8-431b-8c19-9e491da11c58 in RUNNING state
I1212 12:57:17.456408  9148 master.cpp:1305] Agent 
841bed6f-0c12-4c5f-b3af-74557bd90411-S0 at slave(326)@10.3.1.5:56995 
(build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1212 12:57:17.456408  7052 containerizer.cpp:2930] Transitioning the state of 
container 42def7e4-36d8-431b-8c19-9e491da11c58 from RUNNING to DESTROYING
I1212 12:57:17.456408  9148 master.cpp:3364] Disconnecting agent 
841bed6f-0c12-4c5f-b3af-74557bd90411-S0

Re: Review Request 64098: Send status updates when agent re-registers.

2017-12-12 Thread Megha Sharma 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64098/
---

(Updated Dec. 12, 2017, 12:54 p.m.)


Review request for mesos, Ilya Pronin, James Peach, and Jiang Yan Xu.


Changes
---

tested with make check


Bugs: MESOS-6406
https://issues.apache.org/jira/browse/MESOS-6406


Repository: mesos


Description
---

Master will send task status updates to frameworks when an agent
which has been previously removed by the master for being unreachable
or is unknown to the master due to the garbage collection of
the unreachable and gone agents in the registry re-registers.


Diffs (updated)
-

  src/master/master.cpp dfe60ef670edcaefa0c1241df2e2870f650fcf9e 
  src/tests/master_allocator_tests.cpp 3400d70bb0ba564eac43c4639eee0efd4d8059e6 
  src/tests/master_tests.cpp 57eae320a7a398527cd3623c89bf67f319a8e955 
  src/tests/partition_tests.cpp 31ebfe1655438eceae74d72a223df03a9dbd282d 
  src/tests/persistent_volume_tests.cpp 
4aa3c2e8b0f461cd78053707cff8bcb2e6f2b0d7 
  src/tests/slave_recovery_tests.cpp f14c6ef69eb20a03454c8197df79b572a3c6d050 
  src/tests/upgrade_tests.cpp 7f434dbba858f636719eec24e92b306b76430c4c 


Diff: https://reviews.apache.org/r/64098/diff/13/

Changes: https://reviews.apache.org/r/64098/diff/12-13/


Testing
---

with make check


Thanks,

Megha Sharma

Review Request 64536: Removed race from SlaveRecoveryTest.ReconnectExecutor.

2017-12-12 Thread Benno Evers 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64536/
---

Review request for mesos and Alexander Rukletsov.


Repository: mesos


Description
---

Since the executor now sends two status updates in potentially
rapid progression, there was a race where the slave successfully
received a TASK_RUNNING update before shutting down, throwing off
the later checks.


Diffs
-

  src/tests/slave_recovery_tests.cpp 253b0fc2ff7ec1f00937d42636151553c46d5175 


Diff: https://reviews.apache.org/r/64536/diff/1/


Testing
---

`./mesos-tests --gtest_filter="SlaveRecoveryTest/0.ReconnectExecutor" 
--gtest_repeat=500 --gtest_break_on_failure`


Thanks,

Benno Evers

Re: Review Request 64506: Correctly reset slave status when aborting a registration.

2017-12-12 Thread Benno Evers 


> On Dec. 11, 2017, 9:37 p.m., Vinod Kone wrote:
> > src/master/master.cpp
> > Lines 6433-6434 (patched)
> > 
> >
> > Instead of this, I would propose that we have a proper state machine 
> > and transition the agent through that. That would avoid issues like this.

I think its a matter of time allocation: This would probably take 1-2 days, a 
complete refactoring to use a state machine more like 1-2 weeks, and both would 
improve on the current situation. So I wouldn't want to rule out doing this, 
even if I agree that it would be even better to use a state machine.


- Benno


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64506/#review193449
---


On Dec. 12, 2017, 9:57 a.m., Benno Evers wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64506/
> ---
> 
> (Updated Dec. 12, 2017, 9:57 a.m.)
> 
> 
> Review request for mesos and Benjamin Mahler.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Previously, the slave was not erased from the `registering`
> and `reregistering` sets in the master for some code paths
> that would result in a failed (re-)registration attempt.
> 
> This could lead to a state where the reason of the unsuccessful
> (re-)registration attempt is fixed on the agent, but the master
> ignores subsequent attempts because it assumes the previous
> operation is still in progress.
> 
> 
> Diffs
> -
> 
>   src/master/master.cpp b3e074cfe86600793310deb87932fa145e95055d 
>   src/tests/master_tests.cpp 7b087675200fe0ca69319b715997700a91d9f6b2 
> 
> 
> Diff: https://reviews.apache.org/r/64506/diff/1/
> 
> 
> Testing
> ---
> 
> `make check`
> 
> 
> Thanks,
> 
> Benno Evers
> 
>