Re: Review Request 52300: Running task ssh without an instance should pick a random instance
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52300/ --- (Updated Sept. 29, 2016, 5:30 a.m.) Review request for Aurora, Joshua Cohen and Bill Farner. Changes --- When command option is provided, instance must be specified Bugs: AURORA-1110 https://issues.apache.org/jira/browse/AURORA-1110 Repository: aurora Description (updated) --- Running task ssh without an instance should pick the random instance Diffs (updated) - src/main/python/apache/aurora/client/cli/options.py 1245ff15a69a4b4347672f7b556985521e813a00 src/main/python/apache/aurora/client/cli/task.py a8a4edcfda8d879fcc4401bb8f5223de3ee5376d src/test/python/apache/aurora/client/cli/test_task.py d2233b67b37f8b34fae62372aa5d95e0a2e827b5 Diff: https://reviews.apache.org/r/52300/diff/ Testing --- verified in vagrant Thanks, JING CHEN
Re: Review Request 52367: Upgrade to latest pants 1.2.0 pre-release.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52367/#review150813 --- Ship it! Ship It! - Stephan Erb On Sept. 29, 2016, 12:51 vorm., John Sirois wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52367/ > --- > > (Updated Sept. 29, 2016, 12:51 vorm.) > > > Review request for Aurora, Joshua Cohen and Stephan Erb. > > > Bugs: AURORA-1784 > https://issues.apache.org/jira/browse/AURORA-1784 > > > Repository: aurora > > > Description > --- > > This pulls in a fix that went in in 1.2.0dev7 that expands the fetch > timeout for the thrift binary used for our python builds from 1 second > to 30. This should provide relief for a top cause of build flakiness. > > Release notes are here: http://pantsbuild.github.io/notes-master.html > > The latest has dependencies that use cffi which in turn depends on > libffi and so this is added to the Vagrant image. > > Additionally, the mesos slave `--executor_registration_timeout` is set > to 1 minute as a temporary workaround to alleviate flaky e2e test > failures related to: https://issues.apache.org/jira/browse/MESOS-6004. > > Vagrantfile > | 2 +- > build-support/packer/build.sh > | 1 + > examples/vagrant/mesos_config/etc_mesos-slave/executor_registration_timeout > | 1 + > pants.ini > | 2 +- > 4 files changed, 4 insertions(+), 2 deletions(-) > > > Diffs > - > > Vagrantfile 88613bfdcabd6ec305efdbc5d21511ad28710aef > build-support/packer/build.sh 7aed38f6345c18ff6d83bb3028acb88bf2935dc7 > examples/vagrant/mesos_config/etc_mesos-slave/executor_registration_timeout > PRE-CREATION > pants.ini 70db14b6bd8178dee21a6b8c70ea169917435f99 > > Diff: https://reviews.apache.org/r/52367/diff/ > > > Testing > --- > > Locally green: > ``` > ./build-support/jenkins/build.sh > ./build-support/python/make-pycharm-virtualenv > vagrant destroy && ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh > ``` > > > Thanks, > > John Sirois > >
Re: Review Request 52367: Upgrade to latest pants 1.2.0 pre-release.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52367/#review150783 --- Ship it! Ship It! - Zameer Manji On Sept. 28, 2016, 3:51 p.m., John Sirois wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52367/ > --- > > (Updated Sept. 28, 2016, 3:51 p.m.) > > > Review request for Aurora, Joshua Cohen and Stephan Erb. > > > Bugs: AURORA-1784 > https://issues.apache.org/jira/browse/AURORA-1784 > > > Repository: aurora > > > Description > --- > > This pulls in a fix that went in in 1.2.0dev7 that expands the fetch > timeout for the thrift binary used for our python builds from 1 second > to 30. This should provide relief for a top cause of build flakiness. > > Release notes are here: http://pantsbuild.github.io/notes-master.html > > The latest has dependencies that use cffi which in turn depends on > libffi and so this is added to the Vagrant image. > > Additionally, the mesos slave `--executor_registration_timeout` is set > to 1 minute as a temporary workaround to alleviate flaky e2e test > failures related to: https://issues.apache.org/jira/browse/MESOS-6004. > > Vagrantfile > | 2 +- > build-support/packer/build.sh > | 1 + > examples/vagrant/mesos_config/etc_mesos-slave/executor_registration_timeout > | 1 + > pants.ini > | 2 +- > 4 files changed, 4 insertions(+), 2 deletions(-) > > > Diffs > - > > Vagrantfile 88613bfdcabd6ec305efdbc5d21511ad28710aef > build-support/packer/build.sh 7aed38f6345c18ff6d83bb3028acb88bf2935dc7 > examples/vagrant/mesos_config/etc_mesos-slave/executor_registration_timeout > PRE-CREATION > pants.ini 70db14b6bd8178dee21a6b8c70ea169917435f99 > > Diff: https://reviews.apache.org/r/52367/diff/ > > > Testing > --- > > Locally green: > ``` > ./build-support/jenkins/build.sh > ./build-support/python/make-pycharm-virtualenv > vagrant destroy && ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh > ``` > > > Thanks, > > John Sirois > >
Re: Review Request 52367: Upgrade to latest pants 1.2.0 pre-release.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52367/#review150781 --- Ship it! Master (1c1c0a5) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing "@ReviewBot retry" - Aurora ReviewBot On Sept. 28, 2016, 10:51 p.m., John Sirois wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52367/ > --- > > (Updated Sept. 28, 2016, 10:51 p.m.) > > > Review request for Aurora, Joshua Cohen and Stephan Erb. > > > Bugs: AURORA-1784 > https://issues.apache.org/jira/browse/AURORA-1784 > > > Repository: aurora > > > Description > --- > > This pulls in a fix that went in in 1.2.0dev7 that expands the fetch > timeout for the thrift binary used for our python builds from 1 second > to 30. This should provide relief for a top cause of build flakiness. > > Release notes are here: http://pantsbuild.github.io/notes-master.html > > The latest has dependencies that use cffi which in turn depends on > libffi and so this is added to the Vagrant image. > > Additionally, the mesos slave `--executor_registration_timeout` is set > to 1 minute as a temporary workaround to alleviate flaky e2e test > failures related to: https://issues.apache.org/jira/browse/MESOS-6004. > > Vagrantfile > | 2 +- > build-support/packer/build.sh > | 1 + > examples/vagrant/mesos_config/etc_mesos-slave/executor_registration_timeout > | 1 + > pants.ini > | 2 +- > 4 files changed, 4 insertions(+), 2 deletions(-) > > > Diffs > - > > Vagrantfile 88613bfdcabd6ec305efdbc5d21511ad28710aef > build-support/packer/build.sh 7aed38f6345c18ff6d83bb3028acb88bf2935dc7 > examples/vagrant/mesos_config/etc_mesos-slave/executor_registration_timeout > PRE-CREATION > pants.ini 70db14b6bd8178dee21a6b8c70ea169917435f99 > > Diff: https://reviews.apache.org/r/52367/diff/ > > > Testing > --- > > Locally green: > ``` > ./build-support/jenkins/build.sh > ./build-support/python/make-pycharm-virtualenv > vagrant destroy && ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh > ``` > > > Thanks, > > John Sirois > >
Review Request 52367: Upgrade to latest pants 1.2.0 pre-release.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52367/ --- Review request for Aurora, Joshua Cohen and Stephan Erb. Bugs: AURORA-1784 https://issues.apache.org/jira/browse/AURORA-1784 Repository: aurora Description --- This pulls in a fix that went in in 1.2.0dev7 that expands the fetch timeout for the thrift binary used for our python builds from 1 second to 30. This should provide relief for a top cause of build flakiness. Release notes are here: http://pantsbuild.github.io/notes-master.html The latest has dependencies that use cffi which in turn depends on libffi and so this is added to the Vagrant image. Additionally, the mesos slave `--executor_registration_timeout` is set to 1 minute as a temporary workaround to alleviate flaky e2e test failures related to: https://issues.apache.org/jira/browse/MESOS-6004. Vagrantfile | 2 +- build-support/packer/build.sh | 1 + examples/vagrant/mesos_config/etc_mesos-slave/executor_registration_timeout | 1 + pants.ini | 2 +- 4 files changed, 4 insertions(+), 2 deletions(-) Diffs - Vagrantfile 88613bfdcabd6ec305efdbc5d21511ad28710aef build-support/packer/build.sh 7aed38f6345c18ff6d83bb3028acb88bf2935dc7 examples/vagrant/mesos_config/etc_mesos-slave/executor_registration_timeout PRE-CREATION pants.ini 70db14b6bd8178dee21a6b8c70ea169917435f99 Diff: https://reviews.apache.org/r/52367/diff/ Testing --- Locally green: ``` ./build-support/jenkins/build.sh ./build-support/python/make-pycharm-virtualenv vagrant destroy && ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh ``` Thanks, John Sirois
Re: Review Request 51876: Modify executor state transition logic to rely on health checks (if enabled)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51876/#review150769 --- I tried to commit this, but e2e tests hung for me. Kai, can you investigate? - Joshua Cohen On Sept. 28, 2016, 9:07 p.m., Kai Huang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51876/ > --- > > (Updated Sept. 28, 2016, 9:07 p.m.) > > > Review request for Aurora, Joshua Cohen and Zameer Manji. > > > Bugs: AURORA-1225 > https://issues.apache.org/jira/browse/AURORA-1225 > > > Repository: aurora > > > Description > --- > > Modify executor state transition logic to rely on health checks (if enabled). > > [Summary] > Executor needs to start executing user content in STARTING and transition to > RUNNING when a successful required number of health checks is reached. > > This review contains a series of executor changes that implement the health > check driven updates. It gives more context of the design of this feature. > > [Background] > Please see this epic: https://issues.apache.org/jira/browse/AURORA-1225 > and the design doc: > https://docs.google.com/document/d/1ZdgW8S4xMhvKW7iQUX99xZm10NXSxEWR0a-21FP5d94/edit# > for more details and background. > > [Description] > If health check is enabled on vCurrent executor, the health checker will send > a "TASK_RUNNING" message when a successful required number of health checks > is reached within the initial_interval_secs. On the other hand, a > "TASK_FAILED" message was sent if the health checker fails to reach the > required number of health checks within that period, or a maximum number of > failed health check limit is reached after the initital_interval_secs. > > If health check is disabled on the vCurrent executor, it will sends > "TASK_RUNNING" message to scheduler after the thermos runner was started. In > this scenario, the behavior of vCurrent executor will be the same as the > vPrev executor. > > [Change List] > The current change set includes: > 1. Removed the status memoization in ChainedStatusChecker. > 2. Modified the StatusManager to be edge triggered. > 3. Changed the Aurora Executor callback function. > 4. Modified the Health Checker and redefined the meaning > initial_interval_secs. > > > Diffs > - > > src/main/python/apache/aurora/executor/aurora_executor.py > ce5ef680f01831cd89fced8969ae3246c7f60cfd > src/main/python/apache/aurora/executor/common/health_checker.py > 5fc845eceac6f0c048d7489fdc4c672b0c609ea0 > src/main/python/apache/aurora/executor/common/status_checker.py > 795dae2d6b661fc528d952c2315196d94127961f > src/main/python/apache/aurora/executor/status_manager.py > 228a99a05f339e21cd7e769a42b9b2276e7bc3fc > src/test/python/apache/aurora/executor/common/test_health_checker.py > bb6ea69dd94298c5b8cf4d5f06d06eea7790d66e > src/test/python/apache/aurora/executor/common/test_status_checker.py > 5be1981c8c8e88258456adb21aa3ca7c0aa472a7 > src/test/python/apache/aurora/executor/test_status_manager.py > ce4679ba1aa7b42cf0115c943d84663030182d23 > src/test/python/apache/aurora/executor/test_thermos_executor.py > 0bfe9e931f873c9f804f2ba4012e050e1f9fd24e > > Diff: https://reviews.apache.org/r/51876/diff/ > > > Testing > --- > > ./build-support/jenkins/build.sh > > ./pants test.pytest src/test/python/apache/aurora/executor:: > > > Thanks, > > Kai Huang > >
Re: Review Request 51876: Modify executor state transition logic to rely on health checks (if enabled)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51876/#review150764 --- Ship it! Master (1c1c0a5) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing "@ReviewBot retry" - Aurora ReviewBot On Sept. 28, 2016, 9:07 p.m., Kai Huang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51876/ > --- > > (Updated Sept. 28, 2016, 9:07 p.m.) > > > Review request for Aurora, Joshua Cohen and Zameer Manji. > > > Bugs: AURORA-1225 > https://issues.apache.org/jira/browse/AURORA-1225 > > > Repository: aurora > > > Description > --- > > Modify executor state transition logic to rely on health checks (if enabled). > > [Summary] > Executor needs to start executing user content in STARTING and transition to > RUNNING when a successful required number of health checks is reached. > > This review contains a series of executor changes that implement the health > check driven updates. It gives more context of the design of this feature. > > [Background] > Please see this epic: https://issues.apache.org/jira/browse/AURORA-1225 > and the design doc: > https://docs.google.com/document/d/1ZdgW8S4xMhvKW7iQUX99xZm10NXSxEWR0a-21FP5d94/edit# > for more details and background. > > [Description] > If health check is enabled on vCurrent executor, the health checker will send > a "TASK_RUNNING" message when a successful required number of health checks > is reached within the initial_interval_secs. On the other hand, a > "TASK_FAILED" message was sent if the health checker fails to reach the > required number of health checks within that period, or a maximum number of > failed health check limit is reached after the initital_interval_secs. > > If health check is disabled on the vCurrent executor, it will sends > "TASK_RUNNING" message to scheduler after the thermos runner was started. In > this scenario, the behavior of vCurrent executor will be the same as the > vPrev executor. > > [Change List] > The current change set includes: > 1. Removed the status memoization in ChainedStatusChecker. > 2. Modified the StatusManager to be edge triggered. > 3. Changed the Aurora Executor callback function. > 4. Modified the Health Checker and redefined the meaning > initial_interval_secs. > > > Diffs > - > > src/main/python/apache/aurora/executor/aurora_executor.py > ce5ef680f01831cd89fced8969ae3246c7f60cfd > src/main/python/apache/aurora/executor/common/health_checker.py > 5fc845eceac6f0c048d7489fdc4c672b0c609ea0 > src/main/python/apache/aurora/executor/common/status_checker.py > 795dae2d6b661fc528d952c2315196d94127961f > src/main/python/apache/aurora/executor/status_manager.py > 228a99a05f339e21cd7e769a42b9b2276e7bc3fc > src/test/python/apache/aurora/executor/common/test_health_checker.py > bb6ea69dd94298c5b8cf4d5f06d06eea7790d66e > src/test/python/apache/aurora/executor/common/test_status_checker.py > 5be1981c8c8e88258456adb21aa3ca7c0aa472a7 > src/test/python/apache/aurora/executor/test_status_manager.py > ce4679ba1aa7b42cf0115c943d84663030182d23 > src/test/python/apache/aurora/executor/test_thermos_executor.py > 0bfe9e931f873c9f804f2ba4012e050e1f9fd24e > > Diff: https://reviews.apache.org/r/51876/diff/ > > > Testing > --- > > ./build-support/jenkins/build.sh > > ./pants test.pytest src/test/python/apache/aurora/executor:: > > > Thanks, > > Kai Huang > >
Re: Review Request 51876: Modify executor state transition logic to rely on health checks (if enabled)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51876/ --- (Updated Sept. 28, 2016, 9:07 p.m.) Review request for Aurora, Joshua Cohen and Zameer Manji. Changes --- Explicitly check all callback types in StatusManager Bugs: AURORA-1225 https://issues.apache.org/jira/browse/AURORA-1225 Repository: aurora Description --- Modify executor state transition logic to rely on health checks (if enabled). [Summary] Executor needs to start executing user content in STARTING and transition to RUNNING when a successful required number of health checks is reached. This review contains a series of executor changes that implement the health check driven updates. It gives more context of the design of this feature. [Background] Please see this epic: https://issues.apache.org/jira/browse/AURORA-1225 and the design doc: https://docs.google.com/document/d/1ZdgW8S4xMhvKW7iQUX99xZm10NXSxEWR0a-21FP5d94/edit# for more details and background. [Description] If health check is enabled on vCurrent executor, the health checker will send a "TASK_RUNNING" message when a successful required number of health checks is reached within the initial_interval_secs. On the other hand, a "TASK_FAILED" message was sent if the health checker fails to reach the required number of health checks within that period, or a maximum number of failed health check limit is reached after the initital_interval_secs. If health check is disabled on the vCurrent executor, it will sends "TASK_RUNNING" message to scheduler after the thermos runner was started. In this scenario, the behavior of vCurrent executor will be the same as the vPrev executor. [Change List] The current change set includes: 1. Removed the status memoization in ChainedStatusChecker. 2. Modified the StatusManager to be edge triggered. 3. Changed the Aurora Executor callback function. 4. Modified the Health Checker and redefined the meaning initial_interval_secs. Diffs (updated) - src/main/python/apache/aurora/executor/aurora_executor.py ce5ef680f01831cd89fced8969ae3246c7f60cfd src/main/python/apache/aurora/executor/common/health_checker.py 5fc845eceac6f0c048d7489fdc4c672b0c609ea0 src/main/python/apache/aurora/executor/common/status_checker.py 795dae2d6b661fc528d952c2315196d94127961f src/main/python/apache/aurora/executor/status_manager.py 228a99a05f339e21cd7e769a42b9b2276e7bc3fc src/test/python/apache/aurora/executor/common/test_health_checker.py bb6ea69dd94298c5b8cf4d5f06d06eea7790d66e src/test/python/apache/aurora/executor/common/test_status_checker.py 5be1981c8c8e88258456adb21aa3ca7c0aa472a7 src/test/python/apache/aurora/executor/test_status_manager.py ce4679ba1aa7b42cf0115c943d84663030182d23 src/test/python/apache/aurora/executor/test_thermos_executor.py 0bfe9e931f873c9f804f2ba4012e050e1f9fd24e Diff: https://reviews.apache.org/r/51876/diff/ Testing --- ./build-support/jenkins/build.sh ./pants test.pytest src/test/python/apache/aurora/executor:: Thanks, Kai Huang
Re: Review Request 52094: Add min_consecutive_health_checks in HealthCheckConfig
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52094/#review150759 --- Ship it! Ship It! - Joshua Cohen On Sept. 28, 2016, 5 p.m., Kai Huang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52094/ > --- > > (Updated Sept. 28, 2016, 5 p.m.) > > > Review request for Aurora, Joshua Cohen and Zameer Manji. > > > Bugs: AURORA-1224 > https://issues.apache.org/jira/browse/AURORA-1224 > > > Repository: aurora > > > Description > --- > > Add min_consecutive_health_checks to HealthCheckConfig. > > [Summary] > HealthCheckConfig should accept a new configuration value that will tell how > many positive consecutive health checks an instance requires to move from > STARTING to RUNNING. > > [Background] > This review depends on the executor change(AURORA-1225). Please see > https://reviews.apache.org/r/51876/ for more details and background. > > [Change List] > 1. Add a configuration value "min_consecutive_health_checks"(default=1) to > HealthCheckConfig struct. > 2. Modify the default value of watch_secs to be 0. > 3. Add a client-side constraint: > initial_interval_secs >= min_consecutive_health_checks * interval_secs > 4. Update the unit tests for health check config in client/config.py, skip > unit tests related to watch_secs. > > > Diffs > - > > docs/reference/configuration.md f2a0b1873f31e91f3bf0cac6f8448e8130fae688 > src/main/python/apache/aurora/client/config.py > 0186af52f0d7d7e3981ec59bf6a01aafee2bcfb1 > src/main/python/apache/aurora/config/schema/base.py > 845163043b0b7b2f9e7aca14677ca9f094658551 > src/test/python/apache/aurora/client/test_config.py > 5cf68a5145ddf9478baa30453c0bcb73136fa7eb > > Diff: https://reviews.apache.org/r/52094/diff/ > > > Testing > --- > > ./build-support/jenkins/build.sh > > ./pants test.pytest src/test/python/apache/aurora/client:: > > ./pants test.pytest src/test/python/apache/aurora/config:: > > > Thanks, > > Kai Huang > >
Re: Review Request 52343: Add Criteo to Aurora user list
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52343/#review150752 --- Ship it! Ship It! - Zameer Manji On Sept. 28, 2016, 4:46 a.m., Pierre Cheynier wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52343/ > --- > > (Updated Sept. 28, 2016, 4:46 a.m.) > > > Review request for Aurora. > > > Repository: aurora > > > Description > --- > > Add Criteo to Aurora user list > > > Diffs > - > > README.md c314f5ab4e550328caff765deb122486820a08b8 > > Diff: https://reviews.apache.org/r/52343/diff/ > > > Testing > --- > > N/A > > > Thanks, > > Pierre Cheynier > >
Re: Review Request 52343: Add Criteo to Aurora user list
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52343/#review150710 --- Ship it! Master (69cba78) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing "@ReviewBot retry" - Aurora ReviewBot On Sept. 28, 2016, 11:46 a.m., Pierre Cheynier wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52343/ > --- > > (Updated Sept. 28, 2016, 11:46 a.m.) > > > Review request for Aurora. > > > Repository: aurora > > > Description > --- > > Add Criteo to Aurora user list > > > Diffs > - > > README.md c314f5ab4e550328caff765deb122486820a08b8 > > Diff: https://reviews.apache.org/r/52343/diff/ > > > Testing > --- > > N/A > > > Thanks, > > Pierre Cheynier > >
Review Request 52343: Add Criteo to Aurora user list
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52343/ --- Review request for Aurora. Repository: aurora Description --- Add Criteo to Aurora user list Diffs - README.md c314f5ab4e550328caff765deb122486820a08b8 Diff: https://reviews.apache.org/r/52343/diff/ Testing --- N/A Thanks, Pierre Cheynier
