For folks that missed it, here are my notes. Thanks to jie for presenting! (1) Jie presented a containerization benchmark:
https://reviews.apache.org/r/68266/ The motivation to add this was the mount table read issue that came up originally in MESOS-8418 [1]. We only pushed a short term fix for container metrics requests, and the performance of recovering / launching containers is still affected by it until MESOS-9081 [2] is fixed. The benchmark launches N containers, then destroys them and waits for them to terminate. Running it against a 24 core (48 hyperthreaded core) machine for 1000 containers showed roughly a minute for getting 1000 containers launched and a similar amount of time to get them destroyed. We haven't spent time optimizing this so there should be room for improvement. It will be interesting to extend this to compare apples to apples with the numbers in: https://kubernetes.io/blog/2018/05/24/kubernetes-containerd-integration-goes-ga/#pod-startup-latency Please share feedback on the review if you have any thoughts! (2) I gave an update on other performance work: (a) State serving improvements: The parallel serving for the /state endpoint is committed and will land in 1.7.0. There's still more to do here to improve performance by avoiding an extra trip through the master's queue during authorization [3], as well as migrate all reads over to parallel serving. (b) There were some authentication scalability issues discovered during a scale test. Fixes are underway. [4] [5] [6] [7] (c) There remains still some ongoing performance improvements to the allocator. Recently, range subtraction for ports resources was improved [8]. The main focus right now is additional copy elimination of Resources, and the approach that we're attempting is to make Resources copy-on-write [9]. Regardless, 1.7.0 does already include some significant improvements to allocation cycle performance and I'll be sure to cover it in the blog post. (3) Lastly, we had a very brief discussion about jemalloc. James' agreed to share on the existing thread some more details about potential compatibility issues if we were to link our libraries against it. Agenda Doc: https://docs.google.com/document/d/12hWGuzbqyNWc2l1ysbPcXwc0pzHEy 4bodagrlNGCuQU [1] https://issues.apache.org/jira/browse/MESOS-8418 [2] https://issues.apache.org/jira/browse/MESOS-9081 [3] https://issues.apache.org/jira/browse/MESOS-9082 [4] https://issues.apache.org/jira/browse/MESOS-9144 [5] https://issues.apache.org/jira/browse/MESOS-9145 [6] https://issues.apache.org/jira/browse/MESOS-9146 [7] https://issues.apache.org/jira/browse/MESOS-9147 [8] https://issues.apache.org/jira/browse/MESOS-9086 [9] https://issues.apache.org/jira/browse/MESOS-6765