Re: Review Request 57524: Support setting the rootfs on Mesos Containers.

2017-11-16 Thread Jason Lai 


> On Nov. 2, 2017, 9:38 p.m., Mohit Jaggi wrote:
> > Folks, Is this going to get committed?
> 
> Stephan Erb wrote:
> I believe this breaks a few things when enabled (e.g. user creation). I 
> will need to take a closer look next week.
> 
> Jason Lai wrote:
> Thanks for the reply. We have been running this internally at Uber for a 
> while and haven't seen broken user creation happening. Do you mind sharing a 
> bit more specifics about what this could potentially break?

Bump


- Jason


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57524/#review189970
---


On March 13, 2017, 4:36 p.m., Zameer Manji wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57524/
> ---
> 
> (Updated March 13, 2017, 4:36 p.m.)
> 
> 
> Review request for Aurora, Santhosh Kumar Shanmugham and Stephan Erb.
> 
> 
> Bugs: AURORA-1903
> https://issues.apache.org/jira/browse/AURORA-1903
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> The mesos unified containerizer does not support absolute container path 
> mounts if no rootfs is set. This allows operators to switch between our 
> current behaviour (mounting images as a volume) and setting the rootfs. See 
> AURORA-1903 for more detailed analysis.
> 
> 
> Diffs
> -
> 
>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 
> f0b148cd158d61cd89cc51dca9f3fa4c6feb1b49 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorModule.java
>  4dac9757a65e144142d36ee921b85a02a5311fe5 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorSettings.java
>  5c987fd051728486172c8afd34219e86d56f00d5 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> 0d639f66db456858278b0485c91c40975c3b45ac 
>   src/main/java/org/apache/aurora/scheduler/mesos/TestExecutorSettings.java 
> e1cd81e6fbd98f23046e6e775be268be4310c62a 
>   
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
> 93cc34cf8393f969087cd0fd6f577228c00170e9 
> 
> 
> Diff: https://reviews.apache.org/r/57524/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Zameer Manji
> 
>

Re: Review Request 63536: Give jobs the ability to determine how to handle partitions by integrating with new Mesos Partition-Aware APIs

2017-11-16 Thread David McLaughlin 


> On Nov. 16, 2017, 10:01 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java
> > Lines 209 (patched)
> > 
> >
> > I suggest keeping this as a valid transition and allowing the 
> > PartitionManager to deal with it.

See the comment I added to this transition. The TL;DR is it could indefinitely 
block restarts, maintenance and preemption when reschedule=False. I don't think 
this is desirable. Today any such indefinite delays are handled by transient 
task timeouts, but obviously we can't apply those timeouts to PARTITIONED. 

My conclusion after much deliberation (I mention some of this about duplicate 
instances in the design doc) was that it's fine just to move to LOST (which is 
the behavior today). The reasoning I ended up with is that PartitionPolicy is 
intended for keeping tasks *running*, and that partitions encountered when 
tearing tasks down are far less important (or not important at all). 

At the same time, it's a discussion worth fleshing out. The other alternative I 
had considered was to simply ignore PARTITIONED when restarting and wait for 
the transient task timeout to hit. But because we'd only want to do that on 
reschedule=False, this logic would have to live in PartitionManager. But this 
would not solve the problem of actually waiting indefinitely when 
reschedule=False.


> On Nov. 16, 2017, 10:01 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java
> > Lines 281 (patched)
> > 
> >
> > If `FAILED` is in this list, i would expect `FINISHED` as well.

Not sure how I missed that. Thanks!


> On Nov. 16, 2017, 10:01 p.m., Bill Farner wrote:
> > src/main/python/apache/aurora/config/schema/base.py
> > Lines 157 (patched)
> > 
> >
> > I would prefer the name `RescheduleImmediately` (or similar) to make 
> > this self-documenting.

See my reply to Jordan. Disable does not mean this, it's an alias for 
reschedule=False. I lifted the name Disabled straight from the comment in the 
proposal doc, but looks like it's added confusion. I'm thinking of just 
reverting that suggestion and going back to explicit PartitionPolicy.


> On Nov. 16, 2017, 10:01 p.m., Bill Farner wrote:
> > src/test/java/org/apache/aurora/scheduler/state/PartitionManagerTest.java
> > Lines 55 (patched)
> > 
> >
> > (disclaimer: see my longer comment about an integration test before 
> > trying to satisfy other comments in this file)
> > 
> > Have a look at `FakeScheduledExecutor` and see if you find it helpful.  
> > The API is a bit weird, but it aims to simplify this type of test where you 
> > need a `Clock` and a `ScheduledExecutor`.  At the very least, it eliminates 
> > the noise of using `Capture`s.
> 
> David McLaughlin wrote:
> I actually found the capture much easier to read and reason about than 
> the FakeScheduledExecutor, and I thought it led to some pretty clean tests. 
> Curious how others feel.

I had seen the FakeScheduledExecutor and opted not to use it. I actually find 
the capture much easier to read and reason about. Curious how others feel.


> On Nov. 16, 2017, 10:01 p.m., Bill Farner wrote:
> > src/test/java/org/apache/aurora/scheduler/state/PartitionManagerTest.java
> > Lines 55 (patched)
> > 
> >
> > (disclaimer: see my longer comment about an integration test before 
> > trying to satisfy other comments in this file)
> > 
> > Have a look at `FakeScheduledExecutor` and see if you find it helpful.  
> > The API is a bit weird, but it aims to simplify this type of test where you 
> > need a `Clock` and a `ScheduledExecutor`.  At the very least, it eliminates 
> > the noise of using `Capture`s.
> 
> David McLaughlin wrote:
> I had seen the FakeScheduledExecutor and opted not to use it. I actually 
> find the capture much easier to read and reason about. Curious how others 
> feel.

I actually found the capture much easier to read and reason about than the 
FakeScheduledExecutor, and I thought it led to some pretty clean tests. Curious 
how others feel.


> On Nov. 16, 2017, 10:01 p.m., Bill Farner wrote:
> > src/test/java/org/apache/aurora/scheduler/state/PartitionManagerTest.java
> > Lines 109-115 (patched)
> > 
> >
> > Test cases like this lead me to believe that an integration test is 
> > more appropriate.  This test case reads as "nothing happens when a task 
> > with reschedule=false transitions from RUNNING to PARTITIONED".  What we 
> > really mean, though, is that "a task with reschedule=false

Re: Review Request 63536: Give jobs the ability to determine how to handle partitions by integrating with new Mesos Partition-Aware APIs

2017-11-16 Thread Aurora ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63536/#review191261
---


Ship it!




Master (46b1112) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On Nov. 16, 2017, 10:30 p.m., David McLaughlin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63536/
> ---
> 
> (Updated Nov. 16, 2017, 10:30 p.m.)
> 
> 
> Review request for Aurora, Jordan Ly, Santhosh Kumar Shanmugham, and Bill 
> Farner.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This is my prototype code for adding partition-awareness to Aurora. There is 
> a proposal document to accompany this here: 
> https://docs.google.com/document/d/1E3GlsVTJLEMAkDWk2_PTxzkRZcapb8nF_5q5AADQI7g/edit#
> 
> I'd like feedback on the high-level approach before adding unit tests, 
> metrics, logging, etc.
> 
> 
> Diffs
> -
> 
>   RELEASE-NOTES.md d653b797533735fc936a1cff2e005813757e1290 
>   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
> 1d369263d779b549b9304018437f535ddc855966 
>   examples/vagrant/upstart/aurora-scheduler.conf 
> 5ca3caef03b6632cd4dbf47711b1ef183f6a6449 
>   src/main/java/org/apache/aurora/scheduler/base/Conversions.java 
> 33cc012a2cad929b0dd1ce236597b870cfc5aba0 
>   src/main/java/org/apache/aurora/scheduler/base/Jobs.java 
> 8d5f4e57c6b4f847cb74471f246fd0b7dd0cbc36 
>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 
> 7c223eae69503fe1d5bf34c430438637abcbcb9b 
>   
> src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java
>  5e83b2acdde7198d16427d4031e9772f78612554 
>   src/main/java/org/apache/aurora/scheduler/state/PartitionManager.java 
> PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/state/SideEffect.java 
> b91a0852e968b4aa9d74801601671cb61af3648a 
>   src/main/java/org/apache/aurora/scheduler/state/StateManagerImpl.java 
> 9989ed441cd9bc442e6472768880ce7924c3bdd9 
>   src/main/java/org/apache/aurora/scheduler/state/StateModule.java 
> c03fff11ea3a4086f9daaa8b07315006c1b481e4 
>   src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java 
> eb4fe7d78ad1e6ec430c428df527bd0cf3a053c1 
>   src/main/python/apache/aurora/client/cli/jobs.py 
> b79ae56bee0e5692cacf1e66f4a4126b06aaffdc 
>   src/main/python/apache/aurora/config/schema/base.py 
> 18ce826363009e1cc0beac5cce4edf42610d487e 
>   src/main/python/apache/aurora/config/thrift.py 
> bedf8cd6641e1b1a930602791b758d584af4891c 
>   src/test/java/org/apache/aurora/scheduler/base/JobsTest.java 
> 13f656f241a8a9a3d339f4053f165070c2669ef3 
>   src/test/java/org/apache/aurora/scheduler/config/CommandLineTest.java 
> c2d875bb5c393dd95d75251fe86dc649ceba7bd9 
>   
> src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java
>  7b0429109e9a7795e559db264e7737fc55ff0169 
>   src/test/java/org/apache/aurora/scheduler/state/PartitionManagerTest.java 
> PRE-CREATION 
>   src/test/java/org/apache/aurora/scheduler/state/StateManagerImplTest.java 
> 0366cd6e9ddba0c3b9c88ffb50738767a352a17a 
>   src/test/java/org/apache/aurora/scheduler/state/TaskStateMachineTest.java 
> 8d6c3fff0af2df39bb929f760b862a2edf5d6fca 
>   src/test/python/apache/aurora/client/cli/test_task.py 
> 186cb2737ba8e169819b7d54f86a7344a669b6cb 
>   src/test/python/apache/aurora/config/test_thrift.py 
> 7a1567a9b67917072bb0ba3eea5857e968375f4d 
>   src/test/sh/org/apache/aurora/e2e/partition_aware.aurora PRE-CREATION 
>   src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 
> f0819fb7ac758ad1229a76fd9794b393400e9f63 
> 
> 
> Diff: https://reviews.apache.org/r/63536/diff/7/
> 
> 
> Testing
> ---
> 
> Manual testing in Vagrant by stopping and starting the Mesos agent. With 
> three jobs:
> 
> 1) No PartitionPolicy (verified existing behavior of moving from PARTITIONED 
> directly to LOST)
> 2) PartitionPolicy with custom delay_secs (verified sat in PARTITIONED for a 
> while before moving to LOST)
> 3) PartitionPolicy with reschedule=False (verified sat in PARTITIONED 
> indefinitely)
> 
> I also verified tasks are able to transition to various states (back to 
> RUNNING or moving to FAILED, etc.) when you turn the agent back on.
> 
> 
> File Attachments
> 
> 
> Task in PARTITIONED state
>   
> https://reviews.apache.org/media/uploaded/files/2017/11/07/02c7fc72-b11d-4ef9-a86b-914e748cad99__Screen_Shot_2017-11-07_at_11.23.41_AM.png
> Task back into running when partition resolved
>   
> https://reviews.apache.org/media/uploaded/files/2017/11/07/a0413f54-1572-4410-a386-0a22e78fab13__Screen_Shot_2017-11-07_at_11.26.02_AM.png
> Compaction of PARTITIONED

Re: Review Request 63536: Give jobs the ability to determine how to handle partitions by integrating with new Mesos Partition-Aware APIs

2017-11-16 Thread David McLaughlin 


> On Nov. 16, 2017, 10:22 p.m., Jordan Ly wrote:
> > src/main/java/org/apache/aurora/scheduler/state/PartitionManager.java
> > Lines 90-91 (patched)
> > 
> >
> > Can this just be:
> > ```
> > if (stateChange.getNewState().equals(ScheduleStatus.PARTITIONED))
> > ```

partitionPolicy is optional. I'll explain this check with a comment.


> On Nov. 16, 2017, 10:22 p.m., Jordan Ly wrote:
> > src/main/python/apache/aurora/config/schema/base.py
> > Lines 157 (patched)
> > 
> >
> > Can you explain what `Disable` does? If it is `RescheduleImmediately`, 
> > then the default 0 on `PartitionPolicy` is sufficient.

Hmm, Disable might not be a good name for this actually. It's more like 
NeverReschedule. But that is also not a good name because we would give up on 
the partition in certain conditions (user or operator initiated actions).


- David


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63536/#review191256
---


On Nov. 16, 2017, 10:30 p.m., David McLaughlin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63536/
> ---
> 
> (Updated Nov. 16, 2017, 10:30 p.m.)
> 
> 
> Review request for Aurora, Jordan Ly, Santhosh Kumar Shanmugham, and Bill 
> Farner.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This is my prototype code for adding partition-awareness to Aurora. There is 
> a proposal document to accompany this here: 
> https://docs.google.com/document/d/1E3GlsVTJLEMAkDWk2_PTxzkRZcapb8nF_5q5AADQI7g/edit#
> 
> I'd like feedback on the high-level approach before adding unit tests, 
> metrics, logging, etc.
> 
> 
> Diffs
> -
> 
>   RELEASE-NOTES.md d653b797533735fc936a1cff2e005813757e1290 
>   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
> 1d369263d779b549b9304018437f535ddc855966 
>   examples/vagrant/upstart/aurora-scheduler.conf 
> 5ca3caef03b6632cd4dbf47711b1ef183f6a6449 
>   src/main/java/org/apache/aurora/scheduler/base/Conversions.java 
> 33cc012a2cad929b0dd1ce236597b870cfc5aba0 
>   src/main/java/org/apache/aurora/scheduler/base/Jobs.java 
> 8d5f4e57c6b4f847cb74471f246fd0b7dd0cbc36 
>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 
> 7c223eae69503fe1d5bf34c430438637abcbcb9b 
>   
> src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java
>  5e83b2acdde7198d16427d4031e9772f78612554 
>   src/main/java/org/apache/aurora/scheduler/state/PartitionManager.java 
> PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/state/SideEffect.java 
> b91a0852e968b4aa9d74801601671cb61af3648a 
>   src/main/java/org/apache/aurora/scheduler/state/StateManagerImpl.java 
> 9989ed441cd9bc442e6472768880ce7924c3bdd9 
>   src/main/java/org/apache/aurora/scheduler/state/StateModule.java 
> c03fff11ea3a4086f9daaa8b07315006c1b481e4 
>   src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java 
> eb4fe7d78ad1e6ec430c428df527bd0cf3a053c1 
>   src/main/python/apache/aurora/client/cli/jobs.py 
> b79ae56bee0e5692cacf1e66f4a4126b06aaffdc 
>   src/main/python/apache/aurora/config/schema/base.py 
> 18ce826363009e1cc0beac5cce4edf42610d487e 
>   src/main/python/apache/aurora/config/thrift.py 
> bedf8cd6641e1b1a930602791b758d584af4891c 
>   src/test/java/org/apache/aurora/scheduler/base/JobsTest.java 
> 13f656f241a8a9a3d339f4053f165070c2669ef3 
>   src/test/java/org/apache/aurora/scheduler/config/CommandLineTest.java 
> c2d875bb5c393dd95d75251fe86dc649ceba7bd9 
>   
> src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java
>  7b0429109e9a7795e559db264e7737fc55ff0169 
>   src/test/java/org/apache/aurora/scheduler/state/PartitionManagerTest.java 
> PRE-CREATION 
>   src/test/java/org/apache/aurora/scheduler/state/StateManagerImplTest.java 
> 0366cd6e9ddba0c3b9c88ffb50738767a352a17a 
>   src/test/java/org/apache/aurora/scheduler/state/TaskStateMachineTest.java 
> 8d6c3fff0af2df39bb929f760b862a2edf5d6fca 
>   src/test/python/apache/aurora/client/cli/test_task.py 
> 186cb2737ba8e169819b7d54f86a7344a669b6cb 
>   src/test/python/apache/aurora/config/test_thrift.py 
> 7a1567a9b67917072bb0ba3eea5857e968375f4d 
>   src/test/sh/org/apache/aurora/e2e/partition_aware.aurora PRE-CREATION 
>   src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 
> f0819fb7ac758ad1229a76fd9794b393400e9f63 
> 
> 
> Diff: https://reviews.apache.org/r/63536/diff/7/
> 
> 
> Testing
> ---
> 
> Manual testing in Vagrant by stopping and starting the Mesos agent. With 
> three jobs:
> 
> 1) No PartitionPolicy (verified existing behavior of moving from

Re: Review Request 63536: Give jobs the ability to determine how to handle partitions by integrating with new Mesos Partition-Aware APIs

2017-11-16 Thread David McLaughlin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63536/
---

(Updated Nov. 16, 2017, 10:30 p.m.)


Review request for Aurora, Jordan Ly, Santhosh Kumar Shanmugham, and Bill 
Farner.


Changes
---

Add comments to explain the rationale behind why we ignore the PartitionPolicy 
and move straight to LOST when tasks are being 
killed/preempted/restarted/drained.


Repository: aurora


Description
---

This is my prototype code for adding partition-awareness to Aurora. There is a 
proposal document to accompany this here: 
https://docs.google.com/document/d/1E3GlsVTJLEMAkDWk2_PTxzkRZcapb8nF_5q5AADQI7g/edit#

I'd like feedback on the high-level approach before adding unit tests, metrics, 
logging, etc.


Diffs (updated)
-

  RELEASE-NOTES.md d653b797533735fc936a1cff2e005813757e1290 
  api/src/main/thrift/org/apache/aurora/gen/api.thrift 
1d369263d779b549b9304018437f535ddc855966 
  examples/vagrant/upstart/aurora-scheduler.conf 
5ca3caef03b6632cd4dbf47711b1ef183f6a6449 
  src/main/java/org/apache/aurora/scheduler/base/Conversions.java 
33cc012a2cad929b0dd1ce236597b870cfc5aba0 
  src/main/java/org/apache/aurora/scheduler/base/Jobs.java 
8d5f4e57c6b4f847cb74471f246fd0b7dd0cbc36 
  src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 
7c223eae69503fe1d5bf34c430438637abcbcb9b 
  
src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java
 5e83b2acdde7198d16427d4031e9772f78612554 
  src/main/java/org/apache/aurora/scheduler/state/PartitionManager.java 
PRE-CREATION 
  src/main/java/org/apache/aurora/scheduler/state/SideEffect.java 
b91a0852e968b4aa9d74801601671cb61af3648a 
  src/main/java/org/apache/aurora/scheduler/state/StateManagerImpl.java 
9989ed441cd9bc442e6472768880ce7924c3bdd9 
  src/main/java/org/apache/aurora/scheduler/state/StateModule.java 
c03fff11ea3a4086f9daaa8b07315006c1b481e4 
  src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java 
eb4fe7d78ad1e6ec430c428df527bd0cf3a053c1 
  src/main/python/apache/aurora/client/cli/jobs.py 
b79ae56bee0e5692cacf1e66f4a4126b06aaffdc 
  src/main/python/apache/aurora/config/schema/base.py 
18ce826363009e1cc0beac5cce4edf42610d487e 
  src/main/python/apache/aurora/config/thrift.py 
bedf8cd6641e1b1a930602791b758d584af4891c 
  src/test/java/org/apache/aurora/scheduler/base/JobsTest.java 
13f656f241a8a9a3d339f4053f165070c2669ef3 
  src/test/java/org/apache/aurora/scheduler/config/CommandLineTest.java 
c2d875bb5c393dd95d75251fe86dc649ceba7bd9 
  
src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java
 7b0429109e9a7795e559db264e7737fc55ff0169 
  src/test/java/org/apache/aurora/scheduler/state/PartitionManagerTest.java 
PRE-CREATION 
  src/test/java/org/apache/aurora/scheduler/state/StateManagerImplTest.java 
0366cd6e9ddba0c3b9c88ffb50738767a352a17a 
  src/test/java/org/apache/aurora/scheduler/state/TaskStateMachineTest.java 
8d6c3fff0af2df39bb929f760b862a2edf5d6fca 
  src/test/python/apache/aurora/client/cli/test_task.py 
186cb2737ba8e169819b7d54f86a7344a669b6cb 
  src/test/python/apache/aurora/config/test_thrift.py 
7a1567a9b67917072bb0ba3eea5857e968375f4d 
  src/test/sh/org/apache/aurora/e2e/partition_aware.aurora PRE-CREATION 
  src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 
f0819fb7ac758ad1229a76fd9794b393400e9f63 


Diff: https://reviews.apache.org/r/63536/diff/7/

Changes: https://reviews.apache.org/r/63536/diff/6-7/


Testing
---

Manual testing in Vagrant by stopping and starting the Mesos agent. With three 
jobs:

1) No PartitionPolicy (verified existing behavior of moving from PARTITIONED 
directly to LOST)
2) PartitionPolicy with custom delay_secs (verified sat in PARTITIONED for a 
while before moving to LOST)
3) PartitionPolicy with reschedule=False (verified sat in PARTITIONED 
indefinitely)

I also verified tasks are able to transition to various states (back to RUNNING 
or moving to FAILED, etc.) when you turn the agent back on.


File Attachments


Task in PARTITIONED state
  
https://reviews.apache.org/media/uploaded/files/2017/11/07/02c7fc72-b11d-4ef9-a86b-914e748cad99__Screen_Shot_2017-11-07_at_11.23.41_AM.png
Task back into running when partition resolved
  
https://reviews.apache.org/media/uploaded/files/2017/11/07/a0413f54-1572-4410-a386-0a22e78fab13__Screen_Shot_2017-11-07_at_11.26.02_AM.png
Compaction of PARTITIONED cycles (note timestamps)
  
https://reviews.apache.org/media/uploaded/files/2017/11/07/edec32e5-b3ec-4fdc-b93f-5449519805ae__Screen_Shot_2017-11-07_at_11.27.47_AM.png


Thanks,

David McLaughlin

Re: Review Request 63536: Give jobs the ability to determine how to handle partitions by integrating with new Mesos Partition-Aware APIs

2017-11-16 Thread Jordan Ly 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63536/#review191256
---


Fix it, then Ship it!




Overall LGTM! You may want to add some entry to the release notes explaining 
how Mesos should fix some bugs on their end before enabling this.


src/main/java/org/apache/aurora/scheduler/state/PartitionManager.java
Lines 90-91 (patched)


Can this just be:
```
if (stateChange.getNewState().equals(ScheduleStatus.PARTITIONED))
```



src/main/python/apache/aurora/config/schema/base.py
Lines 157 (patched)


Can you explain what `Disable` does? If it is `RescheduleImmediately`, then 
the default 0 on `PartitionPolicy` is sufficient.


- Jordan Ly


On Nov. 16, 2017, 1:54 a.m., David McLaughlin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63536/
> ---
> 
> (Updated Nov. 16, 2017, 1:54 a.m.)
> 
> 
> Review request for Aurora, Jordan Ly, Santhosh Kumar Shanmugham, and Bill 
> Farner.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This is my prototype code for adding partition-awareness to Aurora. There is 
> a proposal document to accompany this here: 
> https://docs.google.com/document/d/1E3GlsVTJLEMAkDWk2_PTxzkRZcapb8nF_5q5AADQI7g/edit#
> 
> I'd like feedback on the high-level approach before adding unit tests, 
> metrics, logging, etc.
> 
> 
> Diffs
> -
> 
>   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
> 1d369263d779b549b9304018437f535ddc855966 
>   examples/vagrant/upstart/aurora-scheduler.conf 
> 5ca3caef03b6632cd4dbf47711b1ef183f6a6449 
>   src/main/java/org/apache/aurora/scheduler/base/Conversions.java 
> 33cc012a2cad929b0dd1ce236597b870cfc5aba0 
>   src/main/java/org/apache/aurora/scheduler/base/Jobs.java 
> 8d5f4e57c6b4f847cb74471f246fd0b7dd0cbc36 
>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 
> 7c223eae69503fe1d5bf34c430438637abcbcb9b 
>   
> src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java
>  5e83b2acdde7198d16427d4031e9772f78612554 
>   src/main/java/org/apache/aurora/scheduler/state/PartitionManager.java 
> PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/state/SideEffect.java 
> b91a0852e968b4aa9d74801601671cb61af3648a 
>   src/main/java/org/apache/aurora/scheduler/state/StateManagerImpl.java 
> 9989ed441cd9bc442e6472768880ce7924c3bdd9 
>   src/main/java/org/apache/aurora/scheduler/state/StateModule.java 
> c03fff11ea3a4086f9daaa8b07315006c1b481e4 
>   src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java 
> eb4fe7d78ad1e6ec430c428df527bd0cf3a053c1 
>   src/main/python/apache/aurora/client/cli/jobs.py 
> b79ae56bee0e5692cacf1e66f4a4126b06aaffdc 
>   src/main/python/apache/aurora/config/schema/base.py 
> 18ce826363009e1cc0beac5cce4edf42610d487e 
>   src/main/python/apache/aurora/config/thrift.py 
> bedf8cd6641e1b1a930602791b758d584af4891c 
>   src/test/java/org/apache/aurora/scheduler/base/JobsTest.java 
> 13f656f241a8a9a3d339f4053f165070c2669ef3 
>   src/test/java/org/apache/aurora/scheduler/config/CommandLineTest.java 
> c2d875bb5c393dd95d75251fe86dc649ceba7bd9 
>   
> src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java
>  7b0429109e9a7795e559db264e7737fc55ff0169 
>   src/test/java/org/apache/aurora/scheduler/state/PartitionManagerTest.java 
> PRE-CREATION 
>   src/test/java/org/apache/aurora/scheduler/state/StateManagerImplTest.java 
> 0366cd6e9ddba0c3b9c88ffb50738767a352a17a 
>   src/test/java/org/apache/aurora/scheduler/state/TaskStateMachineTest.java 
> 8d6c3fff0af2df39bb929f760b862a2edf5d6fca 
>   src/test/python/apache/aurora/client/cli/test_task.py 
> 186cb2737ba8e169819b7d54f86a7344a669b6cb 
>   src/test/python/apache/aurora/config/test_thrift.py 
> 7a1567a9b67917072bb0ba3eea5857e968375f4d 
>   src/test/sh/org/apache/aurora/e2e/partition_aware.aurora PRE-CREATION 
>   src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 
> f0819fb7ac758ad1229a76fd9794b393400e9f63 
> 
> 
> Diff: https://reviews.apache.org/r/63536/diff/6/
> 
> 
> Testing
> ---
> 
> Manual testing in Vagrant by stopping and starting the Mesos agent. With 
> three jobs:
> 
> 1) No PartitionPolicy (verified existing behavior of moving from PARTITIONED 
> directly to LOST)
> 2) PartitionPolicy with custom delay_secs (verified sat in PARTITIONED for a 
> while before moving to LOST)
> 3) PartitionPolicy with reschedule=False (verified sat in PARTITIONED 
> indefinitely)
> 
> I also verified tasks are able to transition to various states (back to 
> RUNNING or moving to FAILED, etc.) when you turn the agent back on.
> 
> 
> File

Re: Review Request 63536: Give jobs the ability to determine how to handle partitions by integrating with new Mesos Partition-Aware APIs

2017-11-16 Thread Bill Farner 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63536/#review191229
---


Fix it, then Ship it!




Looks great overall!  The e2e test is very easy to follow.


src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java
Lines 185 (patched)


Nit: `isPartitionAware` to minutely improve readability.



src/main/java/org/apache/aurora/scheduler/state/PartitionManager.java
Lines 91 (patched)


A `LOG.info` on either side of the branch on task configuration would be 
nice to simplify production debugging.

`Partitioned task will be rescheduled in %s seconds`

`Partitioned task will not be rescheduled`



src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java
Lines 209 (patched)


I suggest keeping this as a valid transition and allowing the 
PartitionManager to deal with it.



src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java
Lines 281 (patched)


If `FAILED` is in this list, i would expect `FINISHED` as well.



src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java
Line 266 (original), 307 (patched)


Nit: order the states consistently where possible.  Makes it easier to scan 
for where states are handled differently.  The prevailing convention in this 
patch seems to be `PARTITIONED` last, which sounds fine to me.



src/main/python/apache/aurora/config/schema/base.py
Lines 157 (patched)


I would prefer the name `RescheduleImmediately` (or similar) to make this 
self-documenting.



src/test/java/org/apache/aurora/scheduler/state/PartitionManagerTest.java
Lines 55 (patched)


(disclaimer: see my longer comment about an integration test before trying 
to satisfy other comments in this file)

Have a look at `FakeScheduledExecutor` and see if you find it helpful.  The 
API is a bit weird, but it aims to simplify this type of test where you need a 
`Clock` and a `ScheduledExecutor`.  At the very least, it eliminates the noise 
of using `Capture`s.



src/test/java/org/apache/aurora/scheduler/state/PartitionManagerTest.java
Lines 57 (patched)


nit: rm line



src/test/java/org/apache/aurora/scheduler/state/PartitionManagerTest.java
Lines 60 (patched)


nit: static; ditto elsewhere



src/test/java/org/apache/aurora/scheduler/state/PartitionManagerTest.java
Lines 74-75 (patched)


can you swap old/new param order?  i was confused by some callers until i 
came up here :-)



src/test/java/org/apache/aurora/scheduler/state/PartitionManagerTest.java
Lines 87 (patched)


nit: newline before



src/test/java/org/apache/aurora/scheduler/state/PartitionManagerTest.java
Lines 109-115 (patched)


Test cases like this lead me to believe that an integration test is more 
appropriate.  This test case reads as "nothing happens when a task with 
reschedule=false transitions from RUNNING to PARTITIONED".  What we really 
mean, though, is that "a task with reschedule=false immediately moves to LOST 
upon transition attempt from RUNNING to PARTITIONED".

With this in mind, i suggest you eliminate this test and move the coverage 
into `StateManagerImplTest`.  I think this is rather natural, since they work 
in unison to manage state transitions.

I don't feel strongly about this, since it is not the convention in the 
codebase, and i don't know what hurdles you may encounter if you try to bundle 
this behavior in `StateManagerImplTest`.



src/test/java/org/apache/aurora/scheduler/state/StateManagerImplTest.java
Lines 431 (patched)


Please phrase this check as:

```java
assertEquals(
  ImmutableList.of(PENDING, ASSIGNED, ...),
  updatedTask.getTaskEvents().stream()
  .map(e -> e.getStatus())
  .collect(Collectors.toList()));
```

This will give greater confidence that the compaction worked as intended.


- Bill Farner


On Nov. 15, 2017, 5:54 p.m., David McLaughlin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63536/
> ---
> 
> (Updated

Re: Review Request 63884: Idempotent Ops and slimmed API for JobUpdateStore

2017-11-16 Thread Aurora ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63884/#review191237
---


Ship it!




Master (46b1112) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On Nov. 16, 2017, 6:20 p.m., Bill Farner wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63884/
> ---
> 
> (Updated Nov. 16, 2017, 6:20 p.m.)
> 
> 
> Review request for Aurora, Jordan Ly and Stephan Erb.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> JobUpdateStore historically had granular APIs in the storage layer to
> minimize unnecessary use of 'expensive' database objects.  The
> in-memory store makes these 'free', so moving business logic out of
> the storage layer is now feasible for performance and pragmatic.
> 
> This patch also introduces idempotent log `Op`s for job updates.
> These ops are dual-read with the existing `Op`s (with the exception of
> job update pruning, which should be safe to ignore).  In a future
> release (and possibly before, with a feature flag), the scheduler
> will write the idempotent `Op`s instead of the non-idempotent variants.
> 
> LogStorage has always had the fundamental expectation that `Op`s are
> idempotent.  The to-be-replaced `Op`s violate this requirement.
> Per `LogStorage.java`:
> This design implies that all mutations must be idempotent and free from
> constraint and thus replayable over newer operations when recovering
> from an old checkpoint.
> 
> 
> Diffs
> -
> 
>   api/src/main/thrift/org/apache/aurora/gen/storage.thrift 
> c692a5f3d715599c949c8fbef8b05c24174829e3 
>   src/jmh/java/org/apache/aurora/benchmark/JobUpdates.java 
> a5d1894a80db13632bf03f593ead46987f1c667b 
>   src/jmh/java/org/apache/aurora/benchmark/UpdateStoreBenchmarks.java 
> c98c514bfc28afaad44bda355720347dd9ee2d3e 
>   
> src/main/java/org/apache/aurora/scheduler/pruning/JobUpdateHistoryPruner.java 
> b2768d98f7c7b0632cd31656640fcb053bcd3be1 
>   src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java 
> 7f8c66cae388571cd3eef2668d1b42e3925dc7e8 
>   src/main/java/org/apache/aurora/scheduler/storage/JobUpdateStore.java 
> b3d906b96f9bfe6e4d3fcc9f3a5d02819a1ef9fe 
>   src/main/java/org/apache/aurora/scheduler/storage/log/LogStorage.java 
> 3ce2c7fc06082ae0ca7a77a33bb7f333212a048c 
>   
> src/main/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl.java 
> 57c483b049c40b56d8c83e3dcce67de00113853b 
>   
> src/main/java/org/apache/aurora/scheduler/storage/log/StreamManagerImpl.java 
> baf2647c54f1f9918139584e5151873a3853e83c 
>   
> src/main/java/org/apache/aurora/scheduler/storage/log/WriteAheadStorage.java 
> 4d051fc4ab280418c7df70924c5e5b496c2ce052 
>   
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemJobUpdateStore.java 
> 826cee9c5c27b80dec55af6fb64fbe9bd72fe03e 
>   src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java 
> bba1161a48738e19f10fcf72395f8d70b481ed13 
>   
> src/main/java/org/apache/aurora/scheduler/updater/JobUpdateControllerImpl.java
>  dc8d11c5003b2637b921b6afa6e2cb5f13102858 
>   
> src/test/java/org/apache/aurora/scheduler/pruning/JobUpdateHistoryPrunerTest.java
>  74db5ecd8f63a2df4e24ca83a03c22e90936bbae 
>   src/test/java/org/apache/aurora/scheduler/quota/QuotaManagerImplTest.java 
> 6be4a9c4145ab38905f153a07a022707186a862c 
>   
> src/test/java/org/apache/aurora/scheduler/storage/AbstractJobUpdateStoreTest.java
>  9fab7aeb99b155cc55c778c9b27e92daa97fe47e 
>   src/test/java/org/apache/aurora/scheduler/storage/log/LogStorageTest.java 
> c208210497454fabd7b3c6d41b4b5f40b1a6bb69 
>   
> src/test/java/org/apache/aurora/scheduler/storage/log/WriteAheadStorageTest.java
>  d5e5c11ad09415e861edfaa521c53fd7904d1bfd 
>   
> src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java
>  39456cfe8b8c8e198a6f8c82e09769bd75db2adc 
>   src/test/java/org/apache/aurora/scheduler/updater/JobUpdaterIT.java 
> 661ce58a840e5a23ff5438517880877cd90d0537 
> 
> 
> Diff: https://reviews.apache.org/r/63884/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Bill Farner
> 
>

Re: Review Request 63750: pants for high-sierra

2017-11-16 Thread Bill Farner 


> On Nov. 14, 2017, 4:13 p.m., Aurora ReviewBot wrote:
> > Master (4fecf1f) is red with this patch.
> >   ./build-support/jenkins/build.sh
> > 
> > WARN] 
> > /home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pants/setup/bootstrap-Linux-x86_64/1.4.0.dev20/local/lib/python2.7/site-packages/pants/build_graph/build_file_aliases.py:52:
> >  DeprecationWarning: DEPRECATED: The `resources=` Python target argument 
> > found on target src/main/python/apache/aurora/client/cli:auroraversion will 
> > be removed in version 1.5.0.dev0.
> >   Depend on resources targets instead.
> >   context_aware_object_factory(parse_context, *args, **kwargs)
> > 
> > WARN] 
> > /home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pants/setup/bootstrap-Linux-x86_64/1.4.0.dev20/local/lib/python2.7/site-packages/pants/engine/legacy/graph.py:173:
> >  DeprecationWarning: DEPRECATED: The `resources=` Python target argument 
> > found on target src/main/python/apache/thermos/observer:observer will be 
> > removed in version 1.5.0.dev0.
> >   Depend on resources targets instead.
> >   return target_cls(build_graph=self, **kwargs)
> > 
> > WARN] 
> > /home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pants/setup/bootstrap-Linux-x86_64/1.4.0.dev20/local/lib/python2.7/site-packages/pants/engine/legacy/graph.py:173:
> >  DeprecationWarning: DEPRECATED: The `resources=` Python target argument 
> > found on target src/main/python/apache/aurora/client/cli:auroraversion will 
> > be removed in version 1.5.0.dev0.
> >   Depend on resources targets instead.
> >   return target_cls(build_graph=self, **kwargs)
> > 
> > 
> > 00:13:35 00:00 [main]
> >(To run a reporting server: ./pants server)
> > 00:13:35 00:00   [setup]
> > 00:13:36 00:01 [parse]
> >Executing tasks in goals: gen -> pyprep -> test
> > 00:13:36 00:01   [gen]
> > 00:13:36 00:01 [thrift-py]
> > 00:13:36 00:01   [cache]
> >No cached artifacts for 4 targets.
> >Invalidated 4 targets.INFO] Attempting to fetch thrift 
> > binary from: 
> > /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/thrift/serve/bin/thrift/linux/x86_64/0.9.1/thrift
> >  ...
> > 
  2% .   180 KB
  4% ..  350 KB
  6% ... 520 KB
  8% 700 KB
 10% .   870 KB
 12% .. 1040 KB
 14% ...1210 KB
 16%    1390 KB
 18% .  1560 KB
 20% .. 1730 KB
 22% ...1900 KB
 24%    2080 KB
 26% .  2250 KB
 28% .. 2420 KB
 30% ...2590 KB
 32%    2770 KB
 34% .  2940 KB
 36% .. 3110 KB
 38% ...3280 KB
 40%    3460 KB
 42% .  3630 KB
 44% .. 3800 KB
 46% ...3970 KB
 48%    4150 KB
 50% .  4320 KB
 52% .. 4490 KB
 54% ...4660 KB
 56%    4840 KB
 58% .  5010 KB
 60% .. 5180 KB
 62% ...5350 KB
 64%    5530 KB
 66% .  5700 KB
 68% .. 5870 KB
 70% ...6040 KB
 72%    6220 KB
 74% .  6390 KB
 76% .. 6560 KB
 78% ...6740 KB
 80%    6910 KB
 82% .  7080 KB
 84% .. 7250 KB
 86% ...7430 KB
 88%

Re: Review Request 63884: Idempotent Ops and slimmed API for JobUpdateStore

2017-11-16 Thread Bill Farner 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63884/#review191206
---



Annotated the diff with some reviewer notes.


src/jmh/java/org/apache/aurora/benchmark/JobUpdates.java
Line 70 (original), 68 (patched)


Events are now saved here, so the subsequent saves are unnecessary.



src/main/java/org/apache/aurora/scheduler/pruning/JobUpdateHistoryPruner.java
Line 42 (original), 54 (patched)


This is now an `AbstractScheduledService`, so the injected executor is 
unneeded.  Additionally, pruning logic was moved here.



src/main/java/org/apache/aurora/scheduler/storage/log/LogStorage.java
Lines 357 (patched)


Reads the new Op type which will be written in the future.



src/main/java/org/apache/aurora/scheduler/storage/log/LogStorage.java
Line 369 (original), 396 (patched)


Copying comment from `WriteAheadStorage.java` for rationale:
> Pruned updates will eventually go away from persisted storage when a new 
snapshot is cut.
> So, persisting pruning attempts is not strictly necessary as the periodic 
pruner will
> provide eventual consistency between volatile and persistent storage upon 
scheduler
> restart. By generating an out of band pruning during log replay the 
consistency is
> achieved sooner without potentially exposing pruned but not yet persisted 
data.

With this behavior in mind, the `PRUNE_JOB_UPDATE_HISTORY` op is no longer 
read nor written.



src/main/java/org/apache/aurora/scheduler/storage/log/StreamManagerImpl.java
Lines 299 (patched)


This will be useful in the future to reduce log write activity for 
`SAVE_JOB_UPDATE` `Op`s.



src/main/java/org/apache/aurora/scheduler/storage/log/WriteAheadStorage.java
Lines 218-223 (original), 213-223 (patched)


This generates a large number of `Op`s for very large updates, but the 
overall space cost should be roughly equivalent to the future mode where the 
`JobUpdateDetails` is written directly.

Reminder - `write()` appends to the in-memory transaction, which is 
persisted as a whole when the transaction completes.



src/main/java/org/apache/aurora/scheduler/storage/mem/MemJobUpdateStore.java
Lines 76-77 (original)


These stats were moved to `JobUpdateControllerImpl`.



src/test/java/org/apache/aurora/scheduler/storage/AbstractJobUpdateStoreTest.java
Lines 294-301 (original)


Removing this test is necessary for idempotency.  We must be able to save 
the same update twice to no effect, and we must be able to save a new version 
of an update to overwrite the original.

The uniqueness constraint is enforced at the non-storage caller, 
`JobUpdateControllerImpl`.



src/test/java/org/apache/aurora/scheduler/storage/AbstractJobUpdateStoreTest.java
Lines 451 (patched)


Whoops, i need to finish this.  Coming in an updated draft.


- Bill Farner


On Nov. 16, 2017, 10:20 a.m., Bill Farner wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63884/
> ---
> 
> (Updated Nov. 16, 2017, 10:20 a.m.)
> 
> 
> Review request for Aurora, Jordan Ly and Stephan Erb.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> JobUpdateStore historically had granular APIs in the storage layer to
> minimize unnecessary use of 'expensive' database objects.  The
> in-memory store makes these 'free', so moving business logic out of
> the storage layer is now feasible for performance and pragmatic.
> 
> This patch also introduces idempotent log `Op`s for job updates.
> These ops are dual-read with the existing `Op`s (with the exception of
> job update pruning, which should be safe to ignore).  In a future
> release (and possibly before, with a feature flag), the scheduler
> will write the idempotent `Op`s instead of the non-idempotent variants.
> 
> LogStorage has always had the fundamental expectation that `Op`s are
> idempotent.  The to-be-replaced `Op`s violate this requirement.
> Per `LogStorage.java`:
> This design implies that all mutations must be idempotent and free from
> constraint and thus replayable over newer operations when recovering
> from an old checkpoint.
> 
> 
> Diffs
> -
> 
>   api/src/main/thrift/org/apache/aurora/gen/storage.thrift 
>

Re: Review Request 63871: Improve Vagrant setup with vagrant-hostmanager

2017-11-16 Thread Aurora ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63871/#review191167
---


Ship it!




Master (46b1112) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On Nov. 16, 2017, 10:05 a.m., Rogier Dikkes wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63871/
> ---
> 
> (Updated Nov. 16, 2017, 10:05 a.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Improve Vagrant setup with vagrant-hostmanager
> 
> Currently the vagrant devcluster is setup with aurora.local in the /etc/hosts 
> file of the guest virtual machine, however the /etc/hosts file on the host 
> machine where Vagrant is running on does not get changed. Result is that 
> clients outside of the Vagrant environment have issues connecting to the 
> devcluster, also the url within the devcluster environment do not work when 
> you use it from your browser. An example of this is the url of the aurora 
> framework in the Mesos master page.
> 
> Found vagrant-hostmanager which is easy to implement, it generates the 
> /etc/hosts and removes entries upon destroying the Vagrant setup. It also has 
> the ability to add entries within the Vagrant environment, for now i left 
> that out of the scope of this issue.
> 
> 
> Diffs
> -
> 
>   Vagrantfile 2c4d23aa1b063d11cbf2d9d24e1b4e7a39437513 
>   docs/getting-started/vagrant.md ca4b704ca5a50b245d62e73321336c79705a3f6e 
> 
> 
> Diff: https://reviews.apache.org/r/63871/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Rogier Dikkes
> 
>

Re: Review Request 63871: Improve Vagrant setup with vagrant-hostmanager

2017-11-16 Thread Rogier Dikkes 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63871/
---

(Updated Nov. 16, 2017, 10:05 a.m.)


Review request for Aurora.


Changes
---

Added description with more information what this is about


Repository: aurora


Description (updated)
---

Improve Vagrant setup with vagrant-hostmanager

Currently the vagrant devcluster is setup with aurora.local in the /etc/hosts 
file of the guest virtual machine, however the /etc/hosts file on the host 
machine where Vagrant is running on does not get changed. Result is that 
clients outside of the Vagrant environment have issues connecting to the 
devcluster, also the url within the devcluster environment do not work when you 
use it from your browser. An example of this is the url of the aurora framework 
in the Mesos master page.

Found vagrant-hostmanager which is easy to implement, it generates the 
/etc/hosts and removes entries upon destroying the Vagrant setup. It also has 
the ability to add entries within the Vagrant environment, for now i left that 
out of the scope of this issue.


Diffs
-

  Vagrantfile 2c4d23aa1b063d11cbf2d9d24e1b4e7a39437513 
  docs/getting-started/vagrant.md ca4b704ca5a50b245d62e73321336c79705a3f6e 


Diff: https://reviews.apache.org/r/63871/diff/1/


Testing
---


Thanks,

Rogier Dikkes

Review Request 63871: Improve Vagrant setup with vagrant-hostmanager

2017-11-16 Thread Rogier Dikkes 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63871/
---

Review request for Aurora.


Repository: aurora


Description
---

Improve Vagrant setup with vagrant-hostmanager


Diffs
-

  Vagrantfile 2c4d23aa1b063d11cbf2d9d24e1b4e7a39437513 
  docs/getting-started/vagrant.md ca4b704ca5a50b245d62e73321336c79705a3f6e 


Diff: https://reviews.apache.org/r/63871/diff/1/


Testing
---


Thanks,

Rogier Dikkes