> On Oct. 8, 2016, 7:40 a.m., Stephan Erb wrote: > > src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql, lines > > 225-240 > > <https://reviews.apache.org/r/52588/diff/4/?file=1527281#file1527281line225> > > > > I am new to the Aurora DB layer, so question ahead: Why do we need this > > _and_ the migration script? Is one of those genreated automatically from > > the other? > > Zameer Manji wrote: > I did not work on our migration code, so I have no idea why we need both. > I imagine we could just replace schema.sql with the migrations, but I believe > there is some nuance there that is hard to capture. I will also defer to Josh > here. > > One is not generated from the other sadly. > > Stephan Erb wrote: > Thanks. Your response is already good enogh for now :)
The gist of it is, when the scheduler starts up, we create the db from schema.sql. After that, when we apply the snapshot from the replicated log, we check to see if there's a DB script present. If so, we drop all tables from the DB and run the DML from that script. That means, in the event that we're starting the scheduler after updating the version, that we've now started up the scheduler with the DB matching the state it was from the previous version. At that point we run the migrations on top of the old DB to apply any necessary schema changes. In the event that the scheduler is started fresh with no snapshot (or the snapshot does not contain a db script) then no migrations are run and the db is used as created from schema.sql. Does that make sense? > On Oct. 8, 2016, 7:40 a.m., Stephan Erb wrote: > > src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java, > > lines 246-248 > > <https://reviews.apache.org/r/52588/diff/4/?file=1527271#file1527271line246> > > > > I believe this is not true. Mesos is ignoring the mode, but the volume > > is still writable. At least it used to be the case. > > > > In our setup, we are mounting a writable `/cache` folder into each > > container as a spill area that is secured by default linux filesystem > > permissions. > > Zameer Manji wrote: > This was a request from Josh, > > I will let you and Josh fight it out here on what we should do. The code > appears to not take into account mode, so I'm not sure if the result is > writable or read only mounts. > > Stephan Erb wrote: > I have verified in Vagrant: The mode is always ignored. Mesos simply > performs the mount and the rest is up to normal filesystem permissions, i.e. > read-write is possible. > > * I have modified the job `examples/jobs/hello_docker_image.aurora` to > begin with a `echo WRITE_WORKED > /cache/maybe-writable`. This jobs uses the > role `www-data` > * Furthermore, I modified the scheduler to launch with ` > -global_container_mounts=/tmp:/cache:ro` > * An `ls -l /tmp` on the agent then shows `-rw-r--r-- 1 www-data > www-data 367 Oct 11 16:30 maybe-writable` I guess on further reflection what it is, is that the result is strictly based on the underlying filesystem permissions and not on the mode passed in. Perhaps the right behavior is to just not support mode at all, since regardless of the value passed in it's the underlying permissions that control the behavior. Alternately we could update the executor to respect the mode, it's a bit tricky to get right though: https://lwn.net/Articles/281157/. - Joshua ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52588/#review151682 ----------------------------------------------------------- On Oct. 7, 2016, 7:19 p.m., Zameer Manji wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52588/ > ----------------------------------------------------------- > > (Updated Oct. 7, 2016, 7:19 p.m.) > > > Review request for Aurora, Joshua Cohen, Santhosh Kumar Shanmugham, and > Stephan Erb. > > > Bugs: AURORA-1107 > https://issues.apache.org/jira/browse/AURORA-1107 > > > Repository: aurora > > > Description > ------- > > This allows users to specify volume mounts for tasks using the unified > containerizer if the operator permits them. This is analogous to enabling > docker > parameters per task and using the `--volume` parameter. > > This does not include the needed DSL changes or an e2e test which will be in a > subsequent diff. > > > Diffs > ----- > > RELEASE-NOTES.md f3dd8bb0f983c560f29ac39824e517c9f145d69e > api/src/main/thrift/org/apache/aurora/gen/api.thrift > 0c74665f750571ccfdeda5a0b71cccbe169716a7 > docs/reference/scheduler-configuration.md > e6b19f01ef276962143dabbaeec4fdf980291a28 > src/main/java/org/apache/aurora/scheduler/app/AppModule.java > c6c2a6d4ec48681378210d1fcb7909fd088d2afb > src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java > 3bd22a0d47cf0793c749b008ccbdf58033c75a11 > > src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java > 701f79c900e1d5794741475333805d1c12af58dd > src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java > 9038c36e4ae05ade273a1c218b7a7e8af218ae39 > src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java > acb44984b4aa38bbe1cac8f805754d55dbe39f3a > src/main/java/org/apache/aurora/scheduler/storage/db/TaskConfigManager.java > e137e57a0b111b9320e6902aa586d7f24e0d5c58 > src/main/java/org/apache/aurora/scheduler/storage/db/TaskConfigMapper.java > 151306a567448c45223e90c8a29ac6ecbc9f6653 > > src/main/java/org/apache/aurora/scheduler/storage/db/migration/V009_CreateContainerVolumesTable.java > PRE-CREATION > > src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/TypeHandlers.java > e30c387239e868daedf496dd886df4d8c00f1fc3 > > src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/VolumeModeTypeHandler.java > PRE-CREATION > src/main/java/org/apache/aurora/scheduler/storage/db/views/DbContainer.java > 8d4d7eca3a17a3b96bbbdec9271503e7e71b3aff > > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskConfigMapper.xml > 3fce25f5e6d180a20dc3ace83f666d47bf32c0c5 > src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql > e943c647af4d0bdb1c733813d019d91942fbbc95 > > src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java > db9f276c1ad26b33ff66f679787ee798bbb69c80 > > src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java > bd052abd65109544be92d3e292f1c2b238c86135 > > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java > 3d07292c83c229cc85d3ad624a7a7a65ef0dcd51 > src/test/java/org/apache/aurora/scheduler/thrift/Fixtures.java > 95b371627d23f7ed3472561f17880bf3c4259b96 > src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java > e578f5a3076b8f1aad86247b815d05c8244dddc8 > > Diff: https://reviews.apache.org/r/52588/diff/ > > > Testing > ------- > > > Thanks, > > Zameer Manji > >
