Re: Review Request 51021: Added GC of unreachable agent metadata from the registry.

2016-09-13 Thread Neil Conway 


> On Sept. 12, 2016, 10:52 p.m., Vinod Kone wrote:
> > src/master/flags.cpp, line 571
> > 
> >
> > do we want to enforce that `registry_max_agent_age` is >= 
> > `registry_gc_interval`? otherwise, users might be confused why old agents 
> > are not being GC'ed?

I considered this. I ended up not doing it because it seemed like additional 
complexity to explain to users why certain configuration values are illegal. 
Configuring `registry_max_agent_age` to be smaller than `registry_gc_interval` 
certainly seems like an usual configuration, but it doesn't seem *wrong* or 
harmful, so I thought I'd allow it. Let me know if disagree.


- Neil


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51021/#review148615
---


On Sept. 12, 2016, 1:48 p.m., Neil Conway wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51021/
> ---
> 
> (Updated Sept. 12, 2016, 1:48 p.m.)
> 
> 
> Review request for mesos and Vinod Kone.
> 
> 
> Bugs: MESOS-5965
> https://issues.apache.org/jira/browse/MESOS-5965
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Information about agents that have been marked unreachable is stored
> in the registry. Since the number of unreachable agents can grow without
> bound, this commit implements a garbage collection scheme. The current
> leading master will periodically examine the registry and discard
> information about unreachable agents according to two criteria:
> `registry_max_agent_age` and `registry_max_agent_count`. The frequency
> with which the master examines the registry is controlled by a third
> parameter, `registry_gc_interval`.
> 
> 
> Diffs
> -
> 
>   src/master/constants.hpp cd80dace80968a1f67a8de5b2c112fb1396e26aa 
>   src/master/flags.hpp c6e85303f60387f42b5e187eaedb6a01000f948f 
>   src/master/flags.cpp 19ae6c1c30a1054b64a9585f325bd0bf943af218 
>   src/master/master.hpp 4992ab0a0bb5babbf6a4fa3e6eff3577590fc879 
>   src/master/master.cpp 1dcce6cd66804990af238176c61aca03bb5c9471 
>   src/master/registry.proto 03c896c39458b54b3966ce32c45eaa33c890ed28 
>   src/tests/partition_tests.cpp f3142ad8d50daafcdb70ad9dbb2772f8ba30db00 
>   src/tests/registrar_tests.cpp b04fc92c66449f1094212a534899e16d5f6f01bd 
> 
> Diff: https://reviews.apache.org/r/51021/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Neil Conway
> 
>

Re: Review Request 51021: Added GC of unreachable agent metadata from the registry.

2016-09-13 Thread Neil Conway 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51021/
---

(Updated Sept. 13, 2016, 12:37 p.m.)


Review request for mesos and Vinod Kone.


Changes
---

Fix typo in comment.


Bugs: MESOS-5965
https://issues.apache.org/jira/browse/MESOS-5965


Repository: mesos


Description
---

Information about agents that have been marked unreachable is stored
in the registry. Since the number of unreachable agents can grow without
bound, this commit implements a garbage collection scheme. The current
leading master will periodically examine the registry and discard
information about unreachable agents according to two criteria:
`registry_max_agent_age` and `registry_max_agent_count`. The frequency
with which the master examines the registry is controlled by a third
parameter, `registry_gc_interval`.


Diffs (updated)
-

  src/master/constants.hpp cd80dace80968a1f67a8de5b2c112fb1396e26aa 
  src/master/flags.hpp c6e85303f60387f42b5e187eaedb6a01000f948f 
  src/master/flags.cpp 19ae6c1c30a1054b64a9585f325bd0bf943af218 
  src/master/master.hpp 4992ab0a0bb5babbf6a4fa3e6eff3577590fc879 
  src/master/master.cpp 1dcce6cd66804990af238176c61aca03bb5c9471 
  src/master/registry.proto 03c896c39458b54b3966ce32c45eaa33c890ed28 
  src/tests/partition_tests.cpp f3142ad8d50daafcdb70ad9dbb2772f8ba30db00 
  src/tests/registrar_tests.cpp b04fc92c66449f1094212a534899e16d5f6f01bd 

Diff: https://reviews.apache.org/r/51021/diff/


Testing
---

make check


Thanks,

Neil Conway

Re: Review Request 51021: Added GC of unreachable agent metadata from the registry.

2016-09-12 Thread Vinod Kone 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51021/#review148615
---


Fix it, then Ship it!




Nice tests!


src/master/flags.cpp (line 571)


do we want to enforce that `registry_max_agent_age` is >= 
`registry_gc_interval`? otherwise, users might be confused why old agents are 
not being GC'ed?



src/tests/partition_tests.cpp (line 1550)


s/update// ?


- Vinod Kone


On Sept. 12, 2016, 1:48 p.m., Neil Conway wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51021/
> ---
> 
> (Updated Sept. 12, 2016, 1:48 p.m.)
> 
> 
> Review request for mesos and Vinod Kone.
> 
> 
> Bugs: MESOS-5965
> https://issues.apache.org/jira/browse/MESOS-5965
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Information about agents that have been marked unreachable is stored
> in the registry. Since the number of unreachable agents can grow without
> bound, this commit implements a garbage collection scheme. The current
> leading master will periodically examine the registry and discard
> information about unreachable agents according to two criteria:
> `registry_max_agent_age` and `registry_max_agent_count`. The frequency
> with which the master examines the registry is controlled by a third
> parameter, `registry_gc_interval`.
> 
> 
> Diffs
> -
> 
>   src/master/constants.hpp cd80dace80968a1f67a8de5b2c112fb1396e26aa 
>   src/master/flags.hpp c6e85303f60387f42b5e187eaedb6a01000f948f 
>   src/master/flags.cpp 19ae6c1c30a1054b64a9585f325bd0bf943af218 
>   src/master/master.hpp 4992ab0a0bb5babbf6a4fa3e6eff3577590fc879 
>   src/master/master.cpp 1dcce6cd66804990af238176c61aca03bb5c9471 
>   src/master/registry.proto 03c896c39458b54b3966ce32c45eaa33c890ed28 
>   src/tests/partition_tests.cpp f3142ad8d50daafcdb70ad9dbb2772f8ba30db00 
>   src/tests/registrar_tests.cpp b04fc92c66449f1094212a534899e16d5f6f01bd 
> 
> Diff: https://reviews.apache.org/r/51021/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Neil Conway
> 
>

Re: Review Request 51021: Added GC of unreachable agent metadata from the registry.

2016-09-12 Thread Neil Conway 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51021/
---

(Updated Sept. 12, 2016, 1:48 p.m.)


Review request for mesos and Vinod Kone.


Changes
---

Tweak test case.


Bugs: MESOS-5965
https://issues.apache.org/jira/browse/MESOS-5965


Repository: mesos


Description
---

Information about agents that have been marked unreachable is stored
in the registry. Since the number of unreachable agents can grow without
bound, this commit implements a garbage collection scheme. The current
leading master will periodically examine the registry and discard
information about unreachable agents according to two criteria:
`registry_max_agent_age` and `registry_max_agent_count`. The frequency
with which the master examines the registry is controlled by a third
parameter, `registry_gc_interval`.


Diffs (updated)
-

  src/master/constants.hpp cd80dace80968a1f67a8de5b2c112fb1396e26aa 
  src/master/flags.hpp c6e85303f60387f42b5e187eaedb6a01000f948f 
  src/master/flags.cpp 19ae6c1c30a1054b64a9585f325bd0bf943af218 
  src/master/master.hpp 4992ab0a0bb5babbf6a4fa3e6eff3577590fc879 
  src/master/master.cpp 1dcce6cd66804990af238176c61aca03bb5c9471 
  src/master/registry.proto 03c896c39458b54b3966ce32c45eaa33c890ed28 
  src/tests/partition_tests.cpp f3142ad8d50daafcdb70ad9dbb2772f8ba30db00 
  src/tests/registrar_tests.cpp b04fc92c66449f1094212a534899e16d5f6f01bd 

Diff: https://reviews.apache.org/r/51021/diff/


Testing
---

make check


Thanks,

Neil Conway

Re: Review Request 51021: Added GC of unreachable agent metadata from the registry.

2016-09-08 Thread Neil Conway 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51021/
---

(Updated Sept. 8, 2016, 3:46 p.m.)


Review request for mesos and Vinod Kone.


Changes
---

Use linkedhashmap, add test for mass removal behavior.


Bugs: MESOS-5965
https://issues.apache.org/jira/browse/MESOS-5965


Repository: mesos


Description
---

Information about agents that have been marked unreachable is stored
in the registry. Since the number of unreachable agents can grow without
bound, this commit implements a garbage collection scheme. The current
leading master will periodically examine the registry and discard
information about unreachable agents according to two criteria:
`registry_max_agent_age` and `registry_max_agent_count`. The frequency
with which the master examines the registry is controlled by a third
parameter, `registry_gc_interval`.


Diffs (updated)
-

  src/master/constants.hpp cd80dace80968a1f67a8de5b2c112fb1396e26aa 
  src/master/flags.hpp c6e85303f60387f42b5e187eaedb6a01000f948f 
  src/master/flags.cpp 19ae6c1c30a1054b64a9585f325bd0bf943af218 
  src/master/master.hpp 4992ab0a0bb5babbf6a4fa3e6eff3577590fc879 
  src/master/master.cpp 1dcce6cd66804990af238176c61aca03bb5c9471 
  src/master/registry.proto 03c896c39458b54b3966ce32c45eaa33c890ed28 
  src/tests/partition_tests.cpp f3142ad8d50daafcdb70ad9dbb2772f8ba30db00 
  src/tests/registrar_tests.cpp b04fc92c66449f1094212a534899e16d5f6f01bd 

Diff: https://reviews.apache.org/r/51021/diff/


Testing
---

make check


Thanks,

Neil Conway

Re: Review Request 51021: Added GC of unreachable agent metadata from the registry.

2016-09-05 Thread Neil Conway 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51021/
---

(Updated Sept. 5, 2016, 2:38 p.m.)


Review request for mesos and Vinod Kone.


Changes
---

Simplify GC scheme.


Bugs: MESOS-5965
https://issues.apache.org/jira/browse/MESOS-5965


Repository: mesos


Description
---

Information about agents that have been marked unreachable is stored
in the registry. Since the number of unreachable agents can grow without
bound, this commit implements a garbage collection scheme. The current
leading master will periodically examine the registry and discard
information about unreachable agents according to two criteria:
`registry_max_agent_age` and `registry_max_agent_count`. The frequency
with which the master examines the registry is controlled by a third
parameter, `registry_gc_interval`.


Diffs (updated)
-

  src/master/constants.hpp cd80dace80968a1f67a8de5b2c112fb1396e26aa 
  src/master/flags.hpp c6e85303f60387f42b5e187eaedb6a01000f948f 
  src/master/flags.cpp 19ae6c1c30a1054b64a9585f325bd0bf943af218 
  src/master/master.hpp c32c7e9d859ef73216354e2c03ecc07d0009b12f 
  src/master/master.cpp b2a19a645528e8fc1fd48f5ac9929d38c9a76b49 
  src/master/registry.proto 03c896c39458b54b3966ce32c45eaa33c890ed28 
  src/tests/partition_tests.cpp f3142ad8d50daafcdb70ad9dbb2772f8ba30db00 
  src/tests/registrar_tests.cpp b04fc92c66449f1094212a534899e16d5f6f01bd 

Diff: https://reviews.apache.org/r/51021/diff/


Testing
---

make check


Thanks,

Neil Conway

Re: Review Request 51021: Added GC of unreachable agent metadata from the registry.

2016-08-29 Thread Neil Conway 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51021/
---

(Updated Aug. 29, 2016, 9:52 a.m.)


Review request for mesos and Vinod Kone.


Changes
---

Rebase.


Bugs: MESOS-5965
https://issues.apache.org/jira/browse/MESOS-5965


Repository: mesos


Description
---

Information about agents that have been marked unreachable is stored
in the registry. Since the number of unreachable agents can grow without
bound, this commit implements a garbage collection scheme. The current
leading master will periodically examine the registry and discard
information about unreachable agents according to two criteria:
`registry_max_agent_age` and `registry_max_agent_count`. The frequency
with which the master examines the registry is controlled by a third
parameter, `registry_gc_interval`.


Diffs (updated)
-

  src/master/constants.hpp cd80dace80968a1f67a8de5b2c112fb1396e26aa 
  src/master/flags.hpp c6e85303f60387f42b5e187eaedb6a01000f948f 
  src/master/flags.cpp 19ae6c1c30a1054b64a9585f325bd0bf943af218 
  src/master/master.hpp b118293823d37bba9b751aac12e0403318ce5736 
  src/master/master.cpp 2b4aff80385d02c71dc3eeff4ddda326c9824ede 
  src/master/registry.proto 03c896c39458b54b3966ce32c45eaa33c890ed28 
  src/tests/partition_tests.cpp f3142ad8d50daafcdb70ad9dbb2772f8ba30db00 
  src/tests/registrar_tests.cpp b04fc92c66449f1094212a534899e16d5f6f01bd 

Diff: https://reviews.apache.org/r/51021/diff/


Testing
---

make check


Thanks,

Neil Conway

Review Request 51021: Added GC of unreachable agent metadata from the registry.

2016-08-25 Thread Neil Conway 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51021/
---

Review request for mesos and Vinod Kone.


Bugs: MESOS-5965
https://issues.apache.org/jira/browse/MESOS-5965


Repository: mesos


Description
---

Information about agents that have been marked unreachable is stored
in the registry. Since the number of unreachable agents can grow without
bound, this commit implements a garbage collection scheme. The current
leading master will periodically examine the registry and discard
information about unreachable agents according to two criteria:
`registry_max_agent_age` and `registry_max_agent_count`. The frequency
with which the master examines the registry is controlled by a third
parameter, `registry_gc_interval`.


Diffs
-

  src/master/constants.hpp cd80dace80968a1f67a8de5b2c112fb1396e26aa 
  src/master/flags.hpp c6e85303f60387f42b5e187eaedb6a01000f948f 
  src/master/flags.cpp 19ae6c1c30a1054b64a9585f325bd0bf943af218 
  src/master/master.hpp 6decff6f4b9c3434de030fd5c06df4c683a7abad 
  src/master/master.cpp 789fb9760a5ea60ce3b7b08f17e36cec69349c60 
  src/master/registry.proto 9bf9998bcf521b962f3fe8741d02620b1907f577 
  src/tests/partition_tests.cpp 0a72b345538ca3b9510fccf38ceb68ac71c2b473 
  src/tests/registrar_tests.cpp 9a71d8fd0c8d8e662a5e364015d144396a0b1a4c 

Diff: https://reviews.apache.org/r/51021/diff/


Testing
---

make check


Thanks,

Neil Conway