Re: Review Request 66490: Implement Mesos docker/volume isolator support for aurora

2018-04-17 Thread Renan DelValle 


> On April 9, 2018, 3:33 p.m., Renan DelValle wrote:
> > src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
> > Lines 277 (patched)
> > 
> >
> > I'm not 100% sure what happens if VolumeType is not set (esp since it's 
> > optional) here but it may crash the scheduler. There should be a check 
> > added to 
> > `src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java`
> > 
> > That way even if someone uses a custom client to schedule a job on 
> > Aurora it'll be able to catch this before it's too late.
> > 
> > It would also be great to add tests to 
> > `src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java`
> >  using this new feature.
> 
> Justin Venus wrote:
> I'll make some more tests and fix what shakes out.
> 
> Justin Venus wrote:
> I have handled the not set case, but I have not yet expanded test cases.

Went through the patch again, we're almost there! 


I just wanted to point out that the better place for that check that I 
mentioned before might be here: 
https://github.com/apache/aurora/blob/master/src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java#L300


- Renan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66490/#review200768
---


On April 13, 2018, 6:52 p.m., Justin Venus wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66490/
> ---
> 
> (Updated April 13, 2018, 6:52 p.m.)
> 
> 
> Review request for Aurora, Renan DelValle and Stephan Erb.
> 
> 
> Bugs: AURORA-1983
> https://issues.apache.org/jira/browse/AURORA-1983
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This change implements the docker/volume isolator as described at 
> http://mesos.apache.org/documentation/latest/isolators/docker-volume/.
> 
> * behind the scenes upgrade configuration to VolumeSource
> * guard against unset enum value while transitioning to VolumeSource
> * replace Iterables.transform w/ Stream interface in MesosTaskFactory
> * Use string for hostPath instead of struct
> * update config thrift tests to check for docker/volume isolator support
> * update mesos task factory to support docker/volume isolator
> * update documentation for docker/volume isolator support
> 
> 
> Diffs
> -
> 
>   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
> ef754e32172e7490a47a13e7b526f243ffa3efeb 
>   docs/reference/configuration.md d4b869b938105ba301fc88d41019af2f1707f6f4 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
>  4073229b74d0e0e7fd31552bd96894ceb8a0971a 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> bcb2bbf882f43d813dd26c746d806e78bae6bcf3 
>   src/main/python/apache/aurora/config/schema/base.py 
> a629bcd1261e5959da0a8458a55545d4e2c2a7a5 
>   src/main/python/apache/aurora/config/thrift.py 
> 6d2dde6e964daa68bf6f0e5bbbffecc5bd8c0431 
>   src/test/python/apache/aurora/config/test_thrift.py 
> 8e1d0e177959af12b97bdd1cd47845b72bc12fe1 
> 
> 
> Diff: https://reviews.apache.org/r/66490/diff/10/
> 
> 
> Testing
> ---
> 
> Tests pass locally
> ```sh
> ./gradlew test
> ./pants test src/test/python::
> ```
> 
> I've deployed this code to a test cluster with rexray, dvdcli, and the agents 
> have `docker/volume` isolation enabled.  I am able to exercise mounting an 
> EBS volume as I would expect into a container.
> 
> 
> Thanks,
> 
> Justin Venus
> 
>

Re: Review Request 66490: Implement Mesos docker/volume isolator support for aurora

2018-04-14 Thread Aurora ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66490/#review201169
---


Ship it!




Master (ad0bc5f) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On April 13, 2018, 6:52 p.m., Justin Venus wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66490/
> ---
> 
> (Updated April 13, 2018, 6:52 p.m.)
> 
> 
> Review request for Aurora, Renan DelValle and Stephan Erb.
> 
> 
> Bugs: AURORA-1983
> https://issues.apache.org/jira/browse/AURORA-1983
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This change implements the docker/volume isolator as described at 
> http://mesos.apache.org/documentation/latest/isolators/docker-volume/.
> 
> * behind the scenes upgrade configuration to VolumeSource
> * guard against unset enum value while transitioning to VolumeSource
> * replace Iterables.transform w/ Stream interface in MesosTaskFactory
> * Use string for hostPath instead of struct
> * update config thrift tests to check for docker/volume isolator support
> * update mesos task factory to support docker/volume isolator
> * update documentation for docker/volume isolator support
> 
> 
> Diffs
> -
> 
>   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
> ef754e32172e7490a47a13e7b526f243ffa3efeb 
>   docs/reference/configuration.md d4b869b938105ba301fc88d41019af2f1707f6f4 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
>  4073229b74d0e0e7fd31552bd96894ceb8a0971a 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> bcb2bbf882f43d813dd26c746d806e78bae6bcf3 
>   src/main/python/apache/aurora/config/schema/base.py 
> a629bcd1261e5959da0a8458a55545d4e2c2a7a5 
>   src/main/python/apache/aurora/config/thrift.py 
> 6d2dde6e964daa68bf6f0e5bbbffecc5bd8c0431 
>   src/test/python/apache/aurora/config/test_thrift.py 
> 8e1d0e177959af12b97bdd1cd47845b72bc12fe1 
> 
> 
> Diff: https://reviews.apache.org/r/66490/diff/10/
> 
> 
> Testing
> ---
> 
> Tests pass locally
> ```sh
> ./gradlew test
> ./pants test src/test/python::
> ```
> 
> I've deployed this code to a test cluster with rexray, dvdcli, and the agents 
> have `docker/volume` isolation enabled.  I am able to exercise mounting an 
> EBS volume as I would expect into a container.
> 
> 
> Thanks,
> 
> Justin Venus
> 
>

Re: Review Request 66490: Implement Mesos docker/volume isolator support for aurora

2018-04-14 Thread Aurora ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66490/#review201167
---



Master (ad0bc5f) is red with this patch.
  ./build-support/jenkins/build.sh

 [191] ./src/main/resources/source-sans-pro.css 1.05 kB {0} [built]
 [223] ./src/main/js/index.js 3.43 kB {0} [built]
+ 268 hidden modules
:processResources
:classes
:jar
:moreStartScripts
:startScripts
:distTar
:distZip
:assemble
:compileTestJavaNote: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: 
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/java/org/apache/aurora/scheduler/storage/durability/DurableStorageTest.java
 uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

:processTestResources
:testClasses
:compileJmhJavaNote: 
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/fakes/FakeSchedulerDriver.java
 uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

:processJmhResources NO-SOURCE
:jmhClasses
:checkstyleJmh
:checkstyleMain[ant:checkstyle] [ERROR] 
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java:281:36:
 Expression can be simplified. [SimplifyBooleanExpression]
 FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':checkstyleMain'.
> Checkstyle rule violations were found. See the report at: 
> file:///home/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/checkstyle/main.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output.

* Get more help at https://help.gradle.org

BUILD FAILED in 4m 17s
32 actionable tasks: 26 executed, 6 up-to-date


I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On April 13, 2018, 6:52 p.m., Justin Venus wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66490/
> ---
> 
> (Updated April 13, 2018, 6:52 p.m.)
> 
> 
> Review request for Aurora, Renan DelValle and Stephan Erb.
> 
> 
> Bugs: AURORA-1983
> https://issues.apache.org/jira/browse/AURORA-1983
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This change implements the docker/volume isolator as described at 
> http://mesos.apache.org/documentation/latest/isolators/docker-volume/.
> 
> * behind the scenes upgrade configuration to VolumeSource
> * guard against unset enum value while transitioning to VolumeSource
> * replace Iterables.transform w/ Stream interface in MesosTaskFactory
> * Use string for hostPath instead of struct
> * update config thrift tests to check for docker/volume isolator support
> * update mesos task factory to support docker/volume isolator
> * update documentation for docker/volume isolator support
> 
> 
> Diffs
> -
> 
>   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
> ef754e32172e7490a47a13e7b526f243ffa3efeb 
>   docs/reference/configuration.md d4b869b938105ba301fc88d41019af2f1707f6f4 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
>  4073229b74d0e0e7fd31552bd96894ceb8a0971a 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> bcb2bbf882f43d813dd26c746d806e78bae6bcf3 
>   src/main/python/apache/aurora/config/schema/base.py 
> a629bcd1261e5959da0a8458a55545d4e2c2a7a5 
>   src/main/python/apache/aurora/config/thrift.py 
> 6d2dde6e964daa68bf6f0e5bbbffecc5bd8c0431 
>   src/test/python/apache/aurora/config/test_thrift.py 
> 8e1d0e177959af12b97bdd1cd47845b72bc12fe1 
> 
> 
> Diff: https://reviews.apache.org/r/66490/diff/9/
> 
> 
> Testing
> ---
> 
> Tests pass locally
> ```sh
> ./gradlew test
> ./pants test src/test/python::
> ```
> 
> I've deployed this code to a test cluster with rexray, dvdcli, and the agents 
> have `docker/volume` isolation enabled.  I am able to exercise mounting an 
> EBS volume as I would expect into a container.
> 
> 
> Thanks,
> 
> Justin Venus
> 
>

Re: Review Request 66490: Implement Mesos docker/volume isolator support for aurora

2018-04-13 Thread Aurora ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66490/#review201157
---



Master (ad0bc5f) is red with this patch.
  ./build-support/jenkins/build.sh

:distTar
:distZip
:assemble
:compileTestJavaNote: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: 
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/java/org/apache/aurora/scheduler/storage/durability/DurableStorageTest.java
 uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

:processTestResources
:testClasses
:compileJmhJavaNote: 
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/fakes/FakeSchedulerDriver.java
 uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

:processJmhResources NO-SOURCE
:jmhClasses
:checkstyleJmh
:checkstyleMain
:checkstyleTest
:licenseJmh UP-TO-DATE
:licenseMain UP-TO-DATE
:licenseTest UP-TO-DATE
:license UP-TO-DATE
:pmdJmh
:pmdMain
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java:281:
  Avoid if (x != y) ..; else ..;
:pmdMain FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':pmdMain'.
> 1 PMD rule violations were found. See the report at: 
> file:///home/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/pmd/main.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output.

* Get more help at https://help.gradle.org

BUILD FAILED in 4m 13s
38 actionable tasks: 29 executed, 9 up-to-date


I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On April 14, 2018, 1:52 a.m., Justin Venus wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66490/
> ---
> 
> (Updated April 14, 2018, 1:52 a.m.)
> 
> 
> Review request for Aurora, Renan DelValle and Stephan Erb.
> 
> 
> Bugs: AURORA-1983
> https://issues.apache.org/jira/browse/AURORA-1983
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This change implements the docker/volume isolator as described at 
> http://mesos.apache.org/documentation/latest/isolators/docker-volume/.
> 
> * behind the scenes upgrade configuration to VolumeSource
> * guard against unset enum value while transitioning to VolumeSource
> * replace Iterables.transform w/ Stream interface in MesosTaskFactory
> * Use string for hostPath instead of struct
> * update config thrift tests to check for docker/volume isolator support
> * update mesos task factory to support docker/volume isolator
> * update documentation for docker/volume isolator support
> 
> 
> Diffs
> -
> 
>   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
> ef754e32172e7490a47a13e7b526f243ffa3efeb 
>   docs/reference/configuration.md d4b869b938105ba301fc88d41019af2f1707f6f4 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
>  4073229b74d0e0e7fd31552bd96894ceb8a0971a 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> bcb2bbf882f43d813dd26c746d806e78bae6bcf3 
>   src/main/python/apache/aurora/config/schema/base.py 
> a629bcd1261e5959da0a8458a55545d4e2c2a7a5 
>   src/main/python/apache/aurora/config/thrift.py 
> 6d2dde6e964daa68bf6f0e5bbbffecc5bd8c0431 
>   src/test/python/apache/aurora/config/test_thrift.py 
> 8e1d0e177959af12b97bdd1cd47845b72bc12fe1 
> 
> 
> Diff: https://reviews.apache.org/r/66490/diff/8/
> 
> 
> Testing
> ---
> 
> Tests pass locally
> ```sh
> ./gradlew test
> ./pants test src/test/python::
> ```
> 
> I've deployed this code to a test cluster with rexray, dvdcli, and the agents 
> have `docker/volume` isolation enabled.  I am able to exercise mounting an 
> EBS volume as I would expect into a container.
> 
> 
> Thanks,
> 
> Justin Venus
> 
>

Re: Review Request 66490: Implement Mesos docker/volume isolator support for aurora

2018-04-13 Thread Aurora ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66490/#review201155
---



Master (ad0bc5f) is red with this patch.
  ./build-support/jenkins/build.sh

 [191] ./src/main/resources/source-sans-pro.css 1.05 kB {0} [built]
 [223] ./src/main/js/index.js 3.43 kB {0} [built]
+ 268 hidden modules
:processResources
:classes
:jar
:moreStartScripts
:startScripts
:distTar
:distZip
:assemble
:compileTestJavaNote: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: 
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/java/org/apache/aurora/scheduler/storage/durability/DurableStorageTest.java
 uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

:processTestResources
:testClasses
:compileJmhJavaNote: 
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/fakes/FakeSchedulerDriver.java
 uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

:processJmhResources NO-SOURCE
:jmhClasses
:checkstyleJmh
:checkstyleMain[ant:checkstyle] [ERROR] 
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java:37:8:
 Unused import - org.apache.aurora.gen.Volume. [UnusedImports]
 FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':checkstyleMain'.
> Checkstyle rule violations were found. See the report at: 
> file:///home/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/checkstyle/main.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output.

* Get more help at https://help.gradle.org

BUILD FAILED in 4m 12s
32 actionable tasks: 26 executed, 6 up-to-date


I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On April 13, 2018, 6:52 p.m., Justin Venus wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66490/
> ---
> 
> (Updated April 13, 2018, 6:52 p.m.)
> 
> 
> Review request for Aurora, Renan DelValle and Stephan Erb.
> 
> 
> Bugs: AURORA-1983
> https://issues.apache.org/jira/browse/AURORA-1983
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This change implements the docker/volume isolator as described at 
> http://mesos.apache.org/documentation/latest/isolators/docker-volume/.
> 
> * behind the scenes upgrade configuration to VolumeSource
> * guard against unset enum value while transitioning to VolumeSource
> * replace Iterables.transform w/ Stream interface in MesosTaskFactory
> * Use string for hostPath instead of struct
> * update config thrift tests to check for docker/volume isolator support
> * update mesos task factory to support docker/volume isolator
> * update documentation for docker/volume isolator support
> 
> 
> Diffs
> -
> 
>   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
> ef754e32172e7490a47a13e7b526f243ffa3efeb 
>   docs/reference/configuration.md d4b869b938105ba301fc88d41019af2f1707f6f4 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
>  4073229b74d0e0e7fd31552bd96894ceb8a0971a 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> bcb2bbf882f43d813dd26c746d806e78bae6bcf3 
>   src/main/python/apache/aurora/config/schema/base.py 
> a629bcd1261e5959da0a8458a55545d4e2c2a7a5 
>   src/main/python/apache/aurora/config/thrift.py 
> 6d2dde6e964daa68bf6f0e5bbbffecc5bd8c0431 
>   src/test/python/apache/aurora/config/test_thrift.py 
> 8e1d0e177959af12b97bdd1cd47845b72bc12fe1 
> 
> 
> Diff: https://reviews.apache.org/r/66490/diff/7/
> 
> 
> Testing
> ---
> 
> Tests pass locally
> ```sh
> ./gradlew test
> ./pants test src/test/python::
> ```
> 
> I've deployed this code to a test cluster with rexray, dvdcli, and the agents 
> have `docker/volume` isolation enabled.  I am able to exercise mounting an 
> EBS volume as I would expect into a container.
> 
> 
> Thanks,
> 
> Justin Venus
> 
>

Re: Review Request 66490: Implement Mesos docker/volume isolator support for aurora

2018-04-13 Thread Justin Venus 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66490/
---

(Updated April 14, 2018, 1:52 a.m.)


Review request for Aurora, Renan DelValle and Stephan Erb.


Changes
---

Transition from Volume.hostPath to Volume.source "@ReviewBot retry"


Bugs: AURORA-1983
https://issues.apache.org/jira/browse/AURORA-1983


Repository: aurora


Description (updated)
---

This change implements the docker/volume isolator as described at 
http://mesos.apache.org/documentation/latest/isolators/docker-volume/.

* behind the scenes upgrade configuration to VolumeSource
* guard against unset enum value while transitioning to VolumeSource
* replace Iterables.transform w/ Stream interface in MesosTaskFactory
* Use string for hostPath instead of struct
* update config thrift tests to check for docker/volume isolator support
* update mesos task factory to support docker/volume isolator
* update documentation for docker/volume isolator support


Diffs (updated)
-

  api/src/main/thrift/org/apache/aurora/gen/api.thrift 
ef754e32172e7490a47a13e7b526f243ffa3efeb 
  docs/reference/configuration.md d4b869b938105ba301fc88d41019af2f1707f6f4 
  
src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
 4073229b74d0e0e7fd31552bd96894ceb8a0971a 
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
bcb2bbf882f43d813dd26c746d806e78bae6bcf3 
  src/main/python/apache/aurora/config/schema/base.py 
a629bcd1261e5959da0a8458a55545d4e2c2a7a5 
  src/main/python/apache/aurora/config/thrift.py 
6d2dde6e964daa68bf6f0e5bbbffecc5bd8c0431 
  src/test/python/apache/aurora/config/test_thrift.py 
8e1d0e177959af12b97bdd1cd47845b72bc12fe1 


Diff: https://reviews.apache.org/r/66490/diff/7/

Changes: https://reviews.apache.org/r/66490/diff/6-7/


Testing
---

Tests pass locally
```sh
./gradlew test
./pants test src/test/python::
```

I've deployed this code to a test cluster with rexray, dvdcli, and the agents 
have `docker/volume` isolation enabled.  I am able to exercise mounting an EBS 
volume as I would expect into a container.


Thanks,

Justin Venus

Re: Review Request 66490: Implement Mesos docker/volume isolator support for aurora

2018-04-12 Thread Justin Venus 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66490/
---

(Updated April 12, 2018, 4:25 p.m.)


Review request for Aurora, Renan DelValle and Stephan Erb.


Changes
---

Replaces all Iterables w/ Java8 Counterparts in MesosTaskFactory.java 
"@ReviewBot retry"


Bugs: AURORA-1983
https://issues.apache.org/jira/browse/AURORA-1983


Repository: aurora


Description
---

This change implements the docker/volume isolator as described at 
http://mesos.apache.org/documentation/latest/isolators/docker-volume/.

* Use string for hostPath instead of struct
* update config thrift tests to check for docker/volume isolator support
* update mesos task factory to support docker/volume isolator
* update documentation for docker/volume isolator support


Diffs (updated)
-

  api/src/main/thrift/org/apache/aurora/gen/api.thrift 
ef754e32172e7490a47a13e7b526f243ffa3efeb 
  docs/reference/configuration.md d4b869b938105ba301fc88d41019af2f1707f6f4 
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
bcb2bbf882f43d813dd26c746d806e78bae6bcf3 
  src/main/python/apache/aurora/config/schema/base.py 
a629bcd1261e5959da0a8458a55545d4e2c2a7a5 
  src/main/python/apache/aurora/config/thrift.py 
6d2dde6e964daa68bf6f0e5bbbffecc5bd8c0431 
  src/test/python/apache/aurora/config/test_thrift.py 
8e1d0e177959af12b97bdd1cd47845b72bc12fe1 


Diff: https://reviews.apache.org/r/66490/diff/6/

Changes: https://reviews.apache.org/r/66490/diff/5-6/


Testing
---

Tests pass locally
```sh
./gradlew test
./pants test src/test/python::
```

I've deployed this code to a test cluster with rexray, dvdcli, and the agents 
have `docker/volume` isolation enabled.  I am able to exercise mounting an EBS 
volume as I would expect into a container.


Thanks,

Justin Venus

Re: Review Request 66490: Implement Mesos docker/volume isolator support for aurora

2018-04-10 Thread Aurora ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66490/#review200867
---


Ship it!




Master (2d6108b) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On April 11, 2018, 12:42 a.m., Justin Venus wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66490/
> ---
> 
> (Updated April 11, 2018, 12:42 a.m.)
> 
> 
> Review request for Aurora, Renan DelValle and Stephan Erb.
> 
> 
> Bugs: AURORA-1983
> https://issues.apache.org/jira/browse/AURORA-1983
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This change implements the docker/volume isolator as described at 
> http://mesos.apache.org/documentation/latest/isolators/docker-volume/.
> 
> * Use string for hostPath instead of struct
> * update config thrift tests to check for docker/volume isolator support
> * update mesos task factory to support docker/volume isolator
> * update documentation for docker/volume isolator support
> 
> 
> Diffs
> -
> 
>   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
> ef754e32172e7490a47a13e7b526f243ffa3efeb 
>   docs/reference/configuration.md d4b869b938105ba301fc88d41019af2f1707f6f4 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> bcb2bbf882f43d813dd26c746d806e78bae6bcf3 
>   src/main/python/apache/aurora/config/schema/base.py 
> a629bcd1261e5959da0a8458a55545d4e2c2a7a5 
>   src/main/python/apache/aurora/config/thrift.py 
> 6d2dde6e964daa68bf6f0e5bbbffecc5bd8c0431 
>   src/test/python/apache/aurora/config/test_thrift.py 
> 8e1d0e177959af12b97bdd1cd47845b72bc12fe1 
> 
> 
> Diff: https://reviews.apache.org/r/66490/diff/5/
> 
> 
> Testing
> ---
> 
> Tests pass locally
> ```sh
> ./gradlew test
> ./pants test src/test/python::
> ```
> 
> I've deployed this code to a test cluster with rexray, dvdcli, and the agents 
> have `docker/volume` isolation enabled.  I am able to exercise mounting an 
> EBS volume as I would expect into a container.
> 
> 
> Thanks,
> 
> Justin Venus
> 
>

Re: Review Request 66490: Implement Mesos docker/volume isolator support for aurora

2018-04-10 Thread Justin Venus 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66490/
---

(Updated April 11, 2018, 12:42 a.m.)


Review request for Aurora, Renan DelValle and Stephan Erb.


Changes
---

Fix: Use string for hostPath instead of struct "@ReviewBot retry"


Bugs: AURORA-1983
https://issues.apache.org/jira/browse/AURORA-1983


Repository: aurora


Description (updated)
---

This change implements the docker/volume isolator as described at 
http://mesos.apache.org/documentation/latest/isolators/docker-volume/.

* Use string for hostPath instead of struct
* update config thrift tests to check for docker/volume isolator support
* update mesos task factory to support docker/volume isolator
* update documentation for docker/volume isolator support


Diffs (updated)
-

  api/src/main/thrift/org/apache/aurora/gen/api.thrift 
ef754e32172e7490a47a13e7b526f243ffa3efeb 
  docs/reference/configuration.md d4b869b938105ba301fc88d41019af2f1707f6f4 
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
bcb2bbf882f43d813dd26c746d806e78bae6bcf3 
  src/main/python/apache/aurora/config/schema/base.py 
a629bcd1261e5959da0a8458a55545d4e2c2a7a5 
  src/main/python/apache/aurora/config/thrift.py 
6d2dde6e964daa68bf6f0e5bbbffecc5bd8c0431 
  src/test/python/apache/aurora/config/test_thrift.py 
8e1d0e177959af12b97bdd1cd47845b72bc12fe1 


Diff: https://reviews.apache.org/r/66490/diff/5/

Changes: https://reviews.apache.org/r/66490/diff/4-5/


Testing
---

Tests pass locally
```sh
./gradlew test
./pants test src/test/python::
```

I've deployed this code to a test cluster with rexray, dvdcli, and the agents 
have `docker/volume` isolation enabled.  I am able to exercise mounting an EBS 
volume as I would expect into a container.


Thanks,

Justin Venus

Re: Review Request 66490: Implement Mesos docker/volume isolator support for aurora

2018-04-10 Thread Renan DelValle 


> On April 9, 2018, 3:33 p.m., Renan DelValle wrote:
> > api/src/main/thrift/org/apache/aurora/gen/api.thrift
> > Lines 1263 (patched)
> > 
> >
> > I think the better solution here would be to make Volume.hostPath 
> > optional and handle hostPath not being set at the Java server side. Maybe 
> > have hostPath mirror to VolumeSource.hostpath while it is being deprecated.
> > 
> > This can be done somewhere in 
> > `src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java`
> 
> Justin Venus wrote:
> Will it break anything to make Volume.hostPath optional?   It's my 
> unstanding that this is implicitly required now.

Yup, I didn't think this one through, sorry about that!

We can leave `Volume.hostpath` as required while making `VolumeSource.hostpath` 
as optional for now. Then, on the scheduler side, if `VolumeSource` is set, we 
use that. If `VolumeSource` is not set, we create a new `VolumeSource` using 
`Volume.hostpath`. And finally, when we create the job, we rely on 
`VolumeSource` as the source of truth.

That way, we can move to deprecate `Volume.hostpath` down the line since it 
becomes a proxy for what `VolumeSource` contains.


- Renan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66490/#review200768
---


On April 6, 2018, 5:04 p.m., Justin Venus wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66490/
> ---
> 
> (Updated April 6, 2018, 5:04 p.m.)
> 
> 
> Review request for Aurora, Renan DelValle and Stephan Erb.
> 
> 
> Bugs: AURORA-1983
> https://issues.apache.org/jira/browse/AURORA-1983
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This change implements the docker/volume isolator as described at 
> http://mesos.apache.org/documentation/latest/isolators/docker-volume/.
>  
> * update config thrift tests to check for docker/volume isolator support
> * update mesos task factory to support docker/volume isolator
> * update documentation for docker/volume isolator support
> 
> 
> Diffs
> -
> 
>   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
> ef754e32172e7490a47a13e7b526f243ffa3efeb 
>   docs/reference/configuration.md d4b869b938105ba301fc88d41019af2f1707f6f4 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> bcb2bbf882f43d813dd26c746d806e78bae6bcf3 
>   src/main/python/apache/aurora/config/schema/base.py 
> a629bcd1261e5959da0a8458a55545d4e2c2a7a5 
>   src/main/python/apache/aurora/config/thrift.py 
> 6d2dde6e964daa68bf6f0e5bbbffecc5bd8c0431 
>   src/test/python/apache/aurora/config/test_thrift.py 
> 8e1d0e177959af12b97bdd1cd47845b72bc12fe1 
> 
> 
> Diff: https://reviews.apache.org/r/66490/diff/4/
> 
> 
> Testing
> ---
> 
> Tests pass locally
> ```sh
> ./gradlew test
> ./pants test src/test/python::
> ```
> 
> I've deployed this code to a test cluster with rexray, dvdcli, and the agents 
> have `docker/volume` isolation enabled.  I am able to exercise mounting an 
> EBS volume as I would expect into a container.
> 
> 
> Thanks,
> 
> Justin Venus
> 
>

Re: Review Request 66490: Implement Mesos docker/volume isolator support for aurora

2018-04-09 Thread Justin Venus 


> On April 9, 2018, 10:33 p.m., Renan DelValle wrote:
> > api/src/main/thrift/org/apache/aurora/gen/api.thrift
> > Lines 203 (patched)
> > 
> >
> > Can we get away with using a string here instead of a struct?

Sure I'll look at refactoring it to use a string.


> On April 9, 2018, 10:33 p.m., Renan DelValle wrote:
> > api/src/main/thrift/org/apache/aurora/gen/api.thrift
> > Lines 1263 (patched)
> > 
> >
> > I think the better solution here would be to make Volume.hostPath 
> > optional and handle hostPath not being set at the Java server side. Maybe 
> > have hostPath mirror to VolumeSource.hostpath while it is being deprecated.
> > 
> > This can be done somewhere in 
> > `src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java`

Will it break anything to make Volume.hostPath optional?   It's my unstanding 
that this is implicitly required now.


> On April 9, 2018, 10:33 p.m., Renan DelValle wrote:
> > src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
> > Lines 277 (patched)
> > 
> >
> > I'm not 100% sure what happens if VolumeType is not set (esp since it's 
> > optional) here but it may crash the scheduler. There should be a check 
> > added to 
> > `src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java`
> > 
> > That way even if someone uses a custom client to schedule a job on 
> > Aurora it'll be able to catch this before it's too late.
> > 
> > It would also be great to add tests to 
> > `src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java`
> >  using this new feature.

I'll make some more tests and fix what shakes out.


> On April 9, 2018, 10:33 p.m., Renan DelValle wrote:
> > src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
> > Lines 284 (patched)
> > 
> >
> > Can we replace all the iterables in this patch with their Java 8 
> > counterpart (e.g. streams)?
> > 
> > The project is currently moving in the direction of deprecating the use 
> > of Guava features that are now replaceable by features in Java 8.

Will do.


- Justin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66490/#review200768
---


On April 7, 2018, 12:04 a.m., Justin Venus wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66490/
> ---
> 
> (Updated April 7, 2018, 12:04 a.m.)
> 
> 
> Review request for Aurora, Renan DelValle and Stephan Erb.
> 
> 
> Bugs: AURORA-1983
> https://issues.apache.org/jira/browse/AURORA-1983
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This change implements the docker/volume isolator as described at 
> http://mesos.apache.org/documentation/latest/isolators/docker-volume/.
>  
> * update config thrift tests to check for docker/volume isolator support
> * update mesos task factory to support docker/volume isolator
> * update documentation for docker/volume isolator support
> 
> 
> Diffs
> -
> 
>   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
> ef754e32172e7490a47a13e7b526f243ffa3efeb 
>   docs/reference/configuration.md d4b869b938105ba301fc88d41019af2f1707f6f4 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> bcb2bbf882f43d813dd26c746d806e78bae6bcf3 
>   src/main/python/apache/aurora/config/schema/base.py 
> a629bcd1261e5959da0a8458a55545d4e2c2a7a5 
>   src/main/python/apache/aurora/config/thrift.py 
> 6d2dde6e964daa68bf6f0e5bbbffecc5bd8c0431 
>   src/test/python/apache/aurora/config/test_thrift.py 
> 8e1d0e177959af12b97bdd1cd47845b72bc12fe1 
> 
> 
> Diff: https://reviews.apache.org/r/66490/diff/4/
> 
> 
> Testing
> ---
> 
> Tests pass locally
> ```sh
> ./gradlew test
> ./pants test src/test/python::
> ```
> 
> I've deployed this code to a test cluster with rexray, dvdcli, and the agents 
> have `docker/volume` isolation enabled.  I am able to exercise mounting an 
> EBS volume as I would expect into a container.
> 
> 
> Thanks,
> 
> Justin Venus
> 
>

Re: Review Request 66490: Implement Mesos docker/volume isolator support for aurora

2018-04-09 Thread Renan DelValle 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66490/#review200768
---



Overall looks good to me. Few fixes here and there. I'll look at the python 
code more thorugoughly next round.


api/src/main/thrift/org/apache/aurora/gen/api.thrift
Lines 203 (patched)


Can we get away with using a string here instead of a struct?



api/src/main/thrift/org/apache/aurora/gen/api.thrift
Lines 1263 (patched)


I think the better solution here would be to make Volume.hostPath optional 
and handle hostPath not being set at the Java server side. Maybe have hostPath 
mirror to VolumeSource.hostpath while it is being deprecated.

This can be done somewhere in 
`src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java`



src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
Lines 277 (patched)


I'm not 100% sure what happens if VolumeType is not set (esp since it's 
optional) here but it may crash the scheduler. There should be a check added to 
`src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java`

That way even if someone uses a custom client to schedule a job on Aurora 
it'll be able to catch this before it's too late.

It would also be great to add tests to 
`src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java` 
using this new feature.



src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
Lines 284 (patched)


Can we replace all the iterables in this patch with their Java 8 
counterpart (e.g. streams)?

The project is currently moving in the direction of deprecating the use of 
Guava features that are now replaceable by features in Java 8.


- Renan DelValle


On April 6, 2018, 5:04 p.m., Justin Venus wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66490/
> ---
> 
> (Updated April 6, 2018, 5:04 p.m.)
> 
> 
> Review request for Aurora, Renan DelValle and Stephan Erb.
> 
> 
> Bugs: AURORA-1983
> https://issues.apache.org/jira/browse/AURORA-1983
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This change implements the docker/volume isolator as described at 
> http://mesos.apache.org/documentation/latest/isolators/docker-volume/.
>  
> * update config thrift tests to check for docker/volume isolator support
> * update mesos task factory to support docker/volume isolator
> * update documentation for docker/volume isolator support
> 
> 
> Diffs
> -
> 
>   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
> ef754e32172e7490a47a13e7b526f243ffa3efeb 
>   docs/reference/configuration.md d4b869b938105ba301fc88d41019af2f1707f6f4 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> bcb2bbf882f43d813dd26c746d806e78bae6bcf3 
>   src/main/python/apache/aurora/config/schema/base.py 
> a629bcd1261e5959da0a8458a55545d4e2c2a7a5 
>   src/main/python/apache/aurora/config/thrift.py 
> 6d2dde6e964daa68bf6f0e5bbbffecc5bd8c0431 
>   src/test/python/apache/aurora/config/test_thrift.py 
> 8e1d0e177959af12b97bdd1cd47845b72bc12fe1 
> 
> 
> Diff: https://reviews.apache.org/r/66490/diff/4/
> 
> 
> Testing
> ---
> 
> Tests pass locally
> ```sh
> ./gradlew test
> ./pants test src/test/python::
> ```
> 
> I've deployed this code to a test cluster with rexray, dvdcli, and the agents 
> have `docker/volume` isolation enabled.  I am able to exercise mounting an 
> EBS volume as I would expect into a container.
> 
> 
> Thanks,
> 
> Justin Venus
> 
>

Re: Review Request 66490: Implement Mesos docker/volume isolator support for aurora

2018-04-06 Thread Aurora ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66490/#review200694
---


Ship it!




Master (70d043e) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On April 7, 2018, 12:04 a.m., Justin Venus wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66490/
> ---
> 
> (Updated April 7, 2018, 12:04 a.m.)
> 
> 
> Review request for Aurora, Renan DelValle and Stephan Erb.
> 
> 
> Bugs: AURORA-1983
> https://issues.apache.org/jira/browse/AURORA-1983
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This change implements the docker/volume isolator as described at 
> http://mesos.apache.org/documentation/latest/isolators/docker-volume/.
>  
> * update config thrift tests to check for docker/volume isolator support
> * update mesos task factory to support docker/volume isolator
> * update documentation for docker/volume isolator support
> 
> 
> Diffs
> -
> 
>   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
> ef754e32172e7490a47a13e7b526f243ffa3efeb 
>   docs/reference/configuration.md d4b869b938105ba301fc88d41019af2f1707f6f4 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> bcb2bbf882f43d813dd26c746d806e78bae6bcf3 
>   src/main/python/apache/aurora/config/schema/base.py 
> a629bcd1261e5959da0a8458a55545d4e2c2a7a5 
>   src/main/python/apache/aurora/config/thrift.py 
> 6d2dde6e964daa68bf6f0e5bbbffecc5bd8c0431 
>   src/test/python/apache/aurora/config/test_thrift.py 
> 8e1d0e177959af12b97bdd1cd47845b72bc12fe1 
> 
> 
> Diff: https://reviews.apache.org/r/66490/diff/4/
> 
> 
> Testing
> ---
> 
> Tests pass locally
> ```sh
> ./gradlew test
> ./pants test src/test/python::
> ```
> 
> I've deployed this code to a test cluster with rexray, dvdcli, and the agents 
> have `docker/volume` isolation enabled.  I am able to exercise mounting an 
> EBS volume as I would expect into a container.
> 
> 
> Thanks,
> 
> Justin Venus
> 
>

Re: Review Request 66490: Implement Mesos docker/volume isolator support for aurora

2018-04-06 Thread Aurora ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66490/#review200692
---



Master (70d043e) is red with this patch.
  ./build-support/jenkins/build.sh

00:44:35 00:00   [setup]
00:44:36 00:01 [parse]
   Executing tasks in goals: lint
00:44:36 00:01   [lint]
00:44:36 00:01 [python-eval]
00:44:36 00:01 [pythonstyle]
00:44:36 00:01   [cache]
   No cached artifacts for 48 targets.
   Invalidated 48 targets.
T302:ERROR   src/test/python/apache/aurora/config/test_thrift.py:218 Expected 2 
blank lines, found 1
 |def test_config_with_host_volumes():

T302:ERROR   src/test/python/apache/aurora/config/test_thrift.py:240 Expected 2 
blank lines, found 1
 |def test_config_with_docker_volumes():

T302:ERROR   src/test/python/apache/aurora/config/test_thrift.py:265 Expected 2 
blank lines, found 1
 |def test_docker_with_parameters():

E302:ERROR   
PythonFile(src/test/python/apache/aurora/config/test_thrift.py):218 expected 2 
blank lines, found 1
 |def test_config_with_host_volumes():

E302:ERROR   
PythonFile(src/test/python/apache/aurora/config/test_thrift.py):240 expected 2 
blank lines, found 1
 |def test_config_with_docker_volumes():

E302:ERROR   
PythonFile(src/test/python/apache/aurora/config/test_thrift.py):265 expected 2 
blank lines, found 1
 |def test_docker_with_parameters():


T302:ERROR   src/main/python/apache/aurora/config/thrift.py:209 Expected 2 
blank lines, found 1
 |def volumes_to_thrift(volumes):

E302:ERROR   PythonFile(src/main/python/apache/aurora/config/thrift.py):209 
expected 2 blank lines, found 1
 |def volumes_to_thrift(volumes):


FAILURE: 8 Python Style issues found. You may try `./pants fmt `


00:44:55 00:20   [complete]
   FAILURE


I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On April 7, 2018, 12:04 a.m., Justin Venus wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66490/
> ---
> 
> (Updated April 7, 2018, 12:04 a.m.)
> 
> 
> Review request for Aurora, Renan DelValle and Stephan Erb.
> 
> 
> Bugs: AURORA-1983
> https://issues.apache.org/jira/browse/AURORA-1983
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This change implements the docker/volume isolator as described at 
> http://mesos.apache.org/documentation/latest/isolators/docker-volume/.
>  
> * update config thrift tests to check for docker/volume isolator support
> * update mesos task factory to support docker/volume isolator
> * update documentation for docker/volume isolator support
> 
> 
> Diffs
> -
> 
>   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
> ef754e32172e7490a47a13e7b526f243ffa3efeb 
>   docs/reference/configuration.md d4b869b938105ba301fc88d41019af2f1707f6f4 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> bcb2bbf882f43d813dd26c746d806e78bae6bcf3 
>   src/main/python/apache/aurora/config/schema/base.py 
> a629bcd1261e5959da0a8458a55545d4e2c2a7a5 
>   src/main/python/apache/aurora/config/thrift.py 
> 6d2dde6e964daa68bf6f0e5bbbffecc5bd8c0431 
>   src/test/python/apache/aurora/config/test_thrift.py 
> 8e1d0e177959af12b97bdd1cd47845b72bc12fe1 
> 
> 
> Diff: https://reviews.apache.org/r/66490/diff/3/
> 
> 
> Testing
> ---
> 
> Tests pass locally
> ```sh
> ./gradlew test
> ./pants test src/test/python::
> ```
> 
> I've deployed this code to a test cluster with rexray, dvdcli, and the agents 
> have `docker/volume` isolation enabled.  I am able to exercise mounting an 
> EBS volume as I would expect into a container.
> 
> 
> Thanks,
> 
> Justin Venus
> 
>

Re: Review Request 66490: Implement Mesos docker/volume isolator support for aurora

2018-04-06 Thread Aurora ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66490/#review200689
---



Master (70d043e) is red with this patch.
  ./build-support/jenkins/build.sh

 [191] ./src/main/resources/source-sans-pro.css 1.05 kB {0} [built]
 [223] ./src/main/js/index.js 3.43 kB {0} [built]
+ 268 hidden modules
:processResources
:classes
:jar
:moreStartScripts
:startScripts
:distTar
:distZip
:assemble
:compileTestJavaNote: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: 
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/java/org/apache/aurora/scheduler/storage/durability/DurableStorageTest.java
 uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

:processTestResources
:testClasses
:compileJmhJavaNote: 
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/fakes/FakeSchedulerDriver.java
 uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

:processJmhResources NO-SOURCE
:jmhClasses
:checkstyleJmh
:checkstyleMain[ant:checkstyle] [ERROR] 
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java:286:
 'lambda arguments' have incorrect indentation level 16, expected level should 
be one of the following: 10, 12. [Indentation]
 FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':checkstyleMain'.
> Checkstyle rule violations were found. See the report at: 
> file:///home/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/checkstyle/main.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output.

* Get more help at https://help.gradle.org

BUILD FAILED in 3m 57s
32 actionable tasks: 26 executed, 6 up-to-date


I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On April 7, 2018, 12:04 a.m., Justin Venus wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66490/
> ---
> 
> (Updated April 7, 2018, 12:04 a.m.)
> 
> 
> Review request for Aurora, Joshua Cohen, Renan DelValle, and Zameer Manji.
> 
> 
> Bugs: AURORA-1983
> https://issues.apache.org/jira/browse/AURORA-1983
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This change implements the docker/volume isolator as described at 
> http://mesos.apache.org/documentation/latest/isolators/docker-volume/.
>  
> * update config thrift tests to check for docker/volume isolator support
> * update mesos task factory to support docker/volume isolator
> * update documentation for docker/volume isolator support
> 
> 
> Diffs
> -
> 
>   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
> ef754e32172e7490a47a13e7b526f243ffa3efeb 
>   docs/reference/configuration.md d4b869b938105ba301fc88d41019af2f1707f6f4 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> bcb2bbf882f43d813dd26c746d806e78bae6bcf3 
>   src/main/python/apache/aurora/config/schema/base.py 
> a629bcd1261e5959da0a8458a55545d4e2c2a7a5 
>   src/main/python/apache/aurora/config/thrift.py 
> 6d2dde6e964daa68bf6f0e5bbbffecc5bd8c0431 
>   src/test/python/apache/aurora/config/test_thrift.py 
> 8e1d0e177959af12b97bdd1cd47845b72bc12fe1 
> 
> 
> Diff: https://reviews.apache.org/r/66490/diff/2/
> 
> 
> Testing
> ---
> 
> Tests pass locally
> ```sh
> ./gradlew test
> ./pants test src/test/python::
> ```
> 
> I've deployed this code to a test cluster with rexray, dvdcli, and the agents 
> have `docker/volume` isolation enabled.  I am able to exercise mounting an 
> EBS volume as I would expect into a container.
> 
> 
> Thanks,
> 
> Justin Venus
> 
>

Re: Review Request 66490: Implement Mesos docker/volume isolator support for aurora

2018-04-06 Thread Justin Venus 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66490/
---

(Updated April 7, 2018, 12:04 a.m.)


Review request for Aurora, Joshua Cohen, Renan DelValle, and Zameer Manji.


Changes
---

fixing style issues "@ReviewBot retry"


Bugs: AURORA-1983
https://issues.apache.org/jira/browse/AURORA-1983


Repository: aurora


Description
---

This change implements the docker/volume isolator as described at 
http://mesos.apache.org/documentation/latest/isolators/docker-volume/.
 
* update config thrift tests to check for docker/volume isolator support
* update mesos task factory to support docker/volume isolator
* update documentation for docker/volume isolator support


Diffs (updated)
-

  api/src/main/thrift/org/apache/aurora/gen/api.thrift 
ef754e32172e7490a47a13e7b526f243ffa3efeb 
  docs/reference/configuration.md d4b869b938105ba301fc88d41019af2f1707f6f4 
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
bcb2bbf882f43d813dd26c746d806e78bae6bcf3 
  src/main/python/apache/aurora/config/schema/base.py 
a629bcd1261e5959da0a8458a55545d4e2c2a7a5 
  src/main/python/apache/aurora/config/thrift.py 
6d2dde6e964daa68bf6f0e5bbbffecc5bd8c0431 
  src/test/python/apache/aurora/config/test_thrift.py 
8e1d0e177959af12b97bdd1cd47845b72bc12fe1 


Diff: https://reviews.apache.org/r/66490/diff/2/

Changes: https://reviews.apache.org/r/66490/diff/1-2/


Testing
---

Tests pass locally
```sh
./gradlew test
./pants test src/test/python::
```

I've deployed this code to a test cluster with rexray, dvdcli, and the agents 
have `docker/volume` isolation enabled.  I am able to exercise mounting an EBS 
volume as I would expect into a container.


Thanks,

Justin Venus

Re: Review Request 66490: Implement Mesos docker/volume isolator support for aurora

2018-04-06 Thread Aurora ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66490/#review200687
---



Master (70d043e) is red with this patch.
  ./build-support/jenkins/build.sh

 [223] ./src/main/js/index.js 3.43 kB {0} [built]
+ 268 hidden modules
:processResources
:classes
:jar
:moreStartScripts
:startScripts
:distTar
:distZip
:assemble
:compileTestJavaNote: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: 
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/java/org/apache/aurora/scheduler/storage/durability/DurableStorageTest.java
 uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

:processTestResources
:testClasses
:compileJmhJavaNote: 
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/fakes/FakeSchedulerDriver.java
 uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

:processJmhResources NO-SOURCE
:jmhClasses
:checkstyleJmh
:checkstyleMain[ant:checkstyle] [ERROR] 
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java:66:15:
 Unused import - org.apache.aurora.gen.apiConstants.DEPRECATED_PATH_SENTINEL. 
[UnusedImports]
[ant:checkstyle] [ERROR] 
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java:285:
 Line is longer than 100 characters (found 115). [LineLength]
 FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':checkstyleMain'.
> Checkstyle rule violations were found. See the report at: 
> file:///home/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/checkstyle/main.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output.

* Get more help at https://help.gradle.org

BUILD FAILED in 4m 15s
32 actionable tasks: 26 executed, 6 up-to-date


I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On April 6, 2018, 11:49 p.m., Justin Venus wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66490/
> ---
> 
> (Updated April 6, 2018, 11:49 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen, Renan DelValle, and Zameer Manji.
> 
> 
> Bugs: AURORA-1983
> https://issues.apache.org/jira/browse/AURORA-1983
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This change implements the docker/volume isolator as described at 
> http://mesos.apache.org/documentation/latest/isolators/docker-volume/.
>  
> * update config thrift tests to check for docker/volume isolator support
> * update mesos task factory to support docker/volume isolator
> * update documentation for docker/volume isolator support
> 
> 
> Diffs
> -
> 
>   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
> ef754e32172e7490a47a13e7b526f243ffa3efeb 
>   docs/reference/configuration.md d4b869b938105ba301fc88d41019af2f1707f6f4 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> bcb2bbf882f43d813dd26c746d806e78bae6bcf3 
>   src/main/python/apache/aurora/config/schema/base.py 
> a629bcd1261e5959da0a8458a55545d4e2c2a7a5 
>   src/main/python/apache/aurora/config/thrift.py 
> 6d2dde6e964daa68bf6f0e5bbbffecc5bd8c0431 
>   src/test/python/apache/aurora/config/test_thrift.py 
> 8e1d0e177959af12b97bdd1cd47845b72bc12fe1 
> 
> 
> Diff: https://reviews.apache.org/r/66490/diff/1/
> 
> 
> Testing
> ---
> 
> Tests pass locally
> ```sh
> ./gradlew test
> ./pants test src/test/python::
> ```
> 
> I've deployed this code to a test cluster with rexray, dvdcli, and the agents 
> have `docker/volume` isolation enabled.  I am able to exercise mounting an 
> EBS volume as I would expect into a container.
> 
> 
> Thanks,
> 
> Justin Venus
> 
>

Review Request 66490: Implement Mesos docker/volume isolator support for aurora

2018-04-06 Thread Justin Venus 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66490/
---

Review request for Aurora, Joshua Cohen, Renan DelValle, and Zameer Manji.


Bugs: AURORA-1983
https://issues.apache.org/jira/browse/AURORA-1983


Repository: aurora


Description
---

This change implements the docker/volume isolator as described at 
http://mesos.apache.org/documentation/latest/isolators/docker-volume/.
 
* update config thrift tests to check for docker/volume isolator support
* update mesos task factory to support docker/volume isolator
* update documentation for docker/volume isolator support


Diffs
-

  api/src/main/thrift/org/apache/aurora/gen/api.thrift 
ef754e32172e7490a47a13e7b526f243ffa3efeb 
  docs/reference/configuration.md d4b869b938105ba301fc88d41019af2f1707f6f4 
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
bcb2bbf882f43d813dd26c746d806e78bae6bcf3 
  src/main/python/apache/aurora/config/schema/base.py 
a629bcd1261e5959da0a8458a55545d4e2c2a7a5 
  src/main/python/apache/aurora/config/thrift.py 
6d2dde6e964daa68bf6f0e5bbbffecc5bd8c0431 
  src/test/python/apache/aurora/config/test_thrift.py 
8e1d0e177959af12b97bdd1cd47845b72bc12fe1 


Diff: https://reviews.apache.org/r/66490/diff/1/


Testing
---

Tests pass locally
```sh
./gradlew test
./pants test src/test/python::
```

I've deployed this code to a test cluster with rexray, dvdcli, and the agents 
have `docker/volume` isolation enabled.  I am able to exercise mounting an EBS 
volume as I would expect into a container.


Thanks,

Justin Venus