> On Dec. 24, 2014, 12:45 a.m., Adam B wrote: > > I'm not sure I buy into the before/after Operation style. Is there any > > previous discussion of alternatives (in a JIRA)?
Adam, unfortunately, there is no specific JIRA for this. However, we can discuss the API in this ticket (https://issues.apache.org/jira/browse/MESOS-2101 or https://issues.apache.org/jira/browse/MESOS-2138). To seed the discussion, I paste the email I sent out ealier here: "In my first iteration, I introduced a concept called Resources::Transformation which transforms a Resources object to a new Resources object. It can be used to represent, for example, the acquisition of a persistent disk from a regular disk (adding DiskInfo), or dynamic reservation (changing reservation type and role). See the following for details: https://reviews.apache.org/r/29018/ https://reviews.apache.org/r/29128/ Now, the question is how the framework specifies a resource transformation during task launch. For acquiring a persistent disk, the framework does not need to explicitly specify a resource transformation because the master can infer those transformations from task resources (see this review for details https://reviews.apache.org/r/28720). However, for dynamic reservation or releasing a persistent disk, the master cannot infer them anymore. We need a way to allow the framework to explicitly specify those resource transformations. Mpark proposed this solution in his review (https://reviews.apache.org/r/29173/). Ben and I made some comments in this review and you may want to take a look. The short summary is that Resources::Transformation is a C++ level abstraction, the framework cannot leverage it to specify transformations like dynamic reservation or persistent disk releasing during task launch. Instead of introducing dynamic reservation specific transformations in https://reviews.apache.org/r/29173/, we propose another solution here (https://reviews.apache.org/r/29274/ and https://reviews.apache.org/r/29380/). The idea is to unify all resource transformations so that the code base looks more consistent. For example, we can kill the existing Resources::Transformation and only use the protobuf introduced in the review. We can add an "apply" method in Resources which applies the transformation to the given resources. The task validation logic can be simplified because we simply just need to apply all the transformations to the offered resources and then check resources usage for each task. One downside of this approach is that the framework needs to explicitly acquire persistent disk resources during task launch (by setting corresponding fields in the launch tasks message). We would like to hear your thoughts on this!" > On Dec. 24, 2014, 12:45 a.m., Adam B wrote: > > include/mesos/mesos.proto, line 429 > > <https://reviews.apache.org/r/29274/diff/1/?file=797605#file797605line429> > > > > s/Operation/Action/? > > Or `Transformation` like you had before? Yeah, I originally propose the name `Transformation`. I think BenM liked the name Operation more since 'acquireXXX'/'releaseXXX' are like "operations". BenM, what do you think if I change it back to Transformation and kill the C++ Resoures::Transformation in favor of this protobuf? > On Dec. 24, 2014, 12:45 a.m., Adam B wrote: > > include/mesos/mesos.proto, line 428 > > <https://reviews.apache.org/r/29274/diff/1/?file=797605#file797605line428> > > > > What about a declarative model: current vs. desired? That's a good idea! I like that! > On Dec. 24, 2014, 12:45 a.m., Adam B wrote: > > include/mesos/mesos.proto, lines 435-437 > > <https://reviews.apache.org/r/29274/diff/1/?file=797605#file797605line435> > > > > Is there going to have to be complex verification? What happens if I > > specify a non-disk resource in before but not after, for a disk-related > > operation? What if a `before` resource doesn't actually exist? > > A declarative approach where one only specifies (an action and) the > > desired outcome would allow the recipient (master) to do whatever is > > necessary to reach the desired outcome. My worry about specifying only the desired outcome is that sometimes it's ambiguate (master has multiple ways to perform the transformation). I think it's the framework who is ultimately responsible for deciding what transformation should be performed, not master. Therefore, I still prefer the 'current/desired' model more because the framework can clearly convey its transforamtion decision. What do you think? I would like to hear comments from others as well. > On Dec. 24, 2014, 12:45 a.m., Adam B wrote: > > include/mesos/mesos.proto, lines 436-437 > > <https://reviews.apache.org/r/29274/diff/1/?file=797605#file797605line436> > > > > So, `Resource` contains an `Operation` which contains repeated > > `Resource`s? But there is no actual Operation field on Resource. > > Looks like ACL is currently the only message that has submessages that > > aren't used by fields of the parent message. > > Perhaps Operation should be a top-level message instead? > > What about Operations on things besides resources? 'Operation' (or 'Transformation') to me is a Resource specific stuff. Putting that in the top level is too vague to me. Maybe we should call it ResourcesTransformation on the top level. What do you think. (But I still prefer putting that nested inside Resource). - Jie ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29274/#review65982 ----------------------------------------------------------- On Dec. 23, 2014, 10:04 p.m., Jie Yu wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/29274/ > ----------------------------------------------------------- > > (Updated Dec. 23, 2014, 10:04 p.m.) > > > Review request for mesos, Adam B, Benjamin Hindman, Ben Mahler, switched to > 'mcypark', and Vinod Kone. > > > Repository: mesos-git > > > Description > ------- > > Added Resource::Operation to mesos.proto per our discussion in > https://reviews.apache.org/r/29173/ > > > Diffs > ----- > > include/mesos/mesos.proto 540071db64961466eb75c779b3ea6863f4594437 > > Diff: https://reviews.apache.org/r/29274/diff/ > > > Testing > ------- > > make check > > > Thanks, > > Jie Yu > >
