----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66139/ -----------------------------------------------------------
(Updated March 20, 2018, 11:41 p.m.) Review request for Aurora, Jordan Ly, Renan DelValle, and Reza Motamedi. Changes ------- Optimize path detection without changing or removing existing code of `ExecutorDetector`. Repository: aurora Description (updated) ------- Profiling indicates that a significant part of the refresh time os spend in `os.path.realpath`. This was introduced in https://reviews.apache.org/r/35580/ to properly handle the `latest` symlink in the Mesos folder layout. This patch takes a slightly different approach to solve this problem based on `os.path.islink`. The latter is faster as it just needs to look at a single folder rather than an entire path. Diffs (updated) ----- src/main/python/apache/aurora/executor/common/path_detector.py ed264d74ef5a5a7aa681a56b340f9b16504a88ad src/test/python/apache/aurora/executor/common/test_path_detector.py 7b5ef0cf552d22d4cfbf3357071de036551026dc Diff: https://reviews.apache.org/r/66139/diff/2/ Changes: https://reviews.apache.org/r/66139/diff/1-2/ Testing (updated) ------- I have tested this build on a node with 55 running tasks and 2004 finished ones. Before this patch: D0320 22:20:44.887248 25771 task_observer.py:142] TaskObserver: finished checkpoint refresh in 0.92s D0320 22:20:50.746316 25771 task_observer.py:142] TaskObserver: finished checkpoint refresh in 0.93s D0320 22:20:56.590157 25771 task_observer.py:142] TaskObserver: finished checkpoint refresh in 0.89s With this patch: D0320 22:18:53.545236 16250 task_observer.py:142] TaskObserver: finished checkpoint refresh in 0.48s D0320 22:18:59.031919 16250 task_observer.py:142] TaskObserver: finished checkpoint refresh in 0.49s D0320 22:19:04.512358 16250 task_observer.py:142] TaskObserver: finished checkpoint refresh in 0.48s Thanks, Stephan Erb
