Mesos / Marathon with Juju
I'm interested in using Juju for deployment of a cassandra cluster, and some microservices, and a front end website; all on a mesos/marathon cluster. Can anyone provide me information if that's feasible today, and what I'd need to know to get started? Thank you! John Weldon -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Coordinating actions in a service
Hi Stuart; I think this is addressed in the proposed work for Actions 2.0 In the current model you'd have to manage all of this yourself. Actions can only be targeted to specific units in the current implementation, so you'd have to manage the distribution outside of actions (or else, as you suggest, some sort of generalised semaphore service, and a way to find all units of a service and queue up actions for all of them and have the actions individually manage themselves whether to run or not) The plan is to allow actions to be targeted at 1) specific units in a service, 2) leaders only, 3) all units in a service, or 4) a subset of units in a service. This would still be a little tricky for your use case, but you could at least manage all the logic in an action targeted at only the leader for example. Cheers, -- John Weldon On Fri, May 8, 2015 at 5:17 AM, Stuart Bishop stuart.bis...@canonical.com wrote: Hi. I have several potentially long running and expensive database operations I'd like to wrap as actions, which will generally be run on just one unit or on all units of the service. The problem I have is running an action on all units of the service. For HA, I need to ensure that, if there is more than one unit, then it may only run on (num_units/2)-1 units at a time. ie. if I fire off an action on all units of a 5 unit cluster, then only two units at a time may run the action and the other units will block until they are done. Leadership is needed to do coordination like this, but I can't see how to use it with actions. The action has no way to request permission from the leader and no way to get a response. Can anyone tell me how to do this with the current model? I don't think it can be done, which I guess makes this a feature request for Actions 2.0. Or perhaps this is another use case for a general locking service providing semaphores (which would need some tricky semantics, since I'd need a semaphore that can be acquired by max(1,(num_units/2)-1) units at a time and num_units might change while waiting for the lock and before releasing it). Or is this just out of scope, and the operator needs to do this sort of coordination themselves? There is plenty of other coordination that can't be embedded in the charm as it is (eg. don't drop an old unit if it is still being used to bootstrap a new unit into the cluster). -- Stuart Bishop stuart.bis...@canonical.com -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Coordinating actions in a service
Actions are not hooks, but they are run in a similar fashion as hooks. I don't know of any restriction keeping actions from communicating with peers before the action completes, but that would be something we'd need to address for Actions 2.0 certainly. I like the concept of metering actions out to units of a service; I would expect that to happen when targeting an action to the service instead of specific units which would require the leader to decide how to fan out the actions. The implementation details for Actions 2.0 are still very conceptual; I'll certainly be calling on you for opinions this go around as we're implementing them :) Cheers, -- John Weldon On Fri, May 8, 2015 at 6:35 AM, Gustavo Niemeyer gust...@niemeyer.net wrote: On Fri, May 8, 2015 at 10:24 AM, John Weldon johnweld...@gmail.com wrote: Hi Stuart; I think this is addressed in the proposed work for Actions 2.0 In the current model you'd have to manage all of this yourself. Actions can only be targeted to specific units in the current implementation, so you'd have to manage the distribution outside of actions (or else, as you suggest, some sort of generalised semaphore service, and a way to find all units of a service and queue up actions for all of them and have the actions individually manage themselves whether to run or not) The plan is to allow actions to be targeted at 1) specific units in a service, 2) leaders only, 3) all units in a service, or 4) a subset of units in a service. This would still be a little tricky for your use case, but you could at least manage all the logic in an action targeted at only the leader for example. None of these seem to fix the issue, though. The requirement is to execute on all units, but not at the same time. Also, the leader has no way to communicate with peer units without the hook terminating, right? There should be a way to postpone the result of an action to a moment past the end of the hook, but I actually think the proper way to fix this is to avoid the avalanche in the first place, by default: when dispatching an action to all units of a service, roll out in a sane way rather than doing all at once. gustavo @ http://niemeyer.net -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Reviewboard - dependant branches
Can someone remind me of the process for proposing branches that depend on another branch that is still in review? I know it's been discussed here and on irc before, but I don't recall the specifics. I think that the steps are: 1. Just propose the branch in github, which will trigger the automatic reviewboard integration 2. run some rbt command line invocation to update reviewboard with the upstream branch name I just don't recall the exact invocation for #2 and I don't want to experiment and muck something up :) Cheers, -- John Weldon -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Too space, or not two space
+1 to the Third Way (and to single space) -- John Weldon On Tue, Nov 18, 2014 at 4:15 PM, Jesse Meek jesse.m...@canonical.com wrote: I take that as +1 from Dave. On 19/11/14 13:13, David Cheney wrote: To quote Butterick's http://practicaltypography.com/one-space-between-sentences.html I think two spaces look better so that’s what I’m going to use. I’m telling you the rule. If you want to put personal taste ahead of the rule, I can’t stop you. But personal taste doesn’t make the rule evaporate. It’s like saying “I don’t like how subjunctive-mood verbs sound, so I’m never going to use them.” On Wed, Nov 19, 2014 at 11:10 AM, Jesse Meek jesse.m...@canonical.com wrote: I'm coming across double spaces after a full stop in comments. Some consider this an error, others an intentional style that makes the comment cleaner to read. Yes, it's a minor point but in the name of consistency and avoiding future review ping pong on the issue, I'm calling for a vote: Should we have a single space after full stops in comments? http://xkcd.com/1285/ Please reply +1 for yes to one space, -1 for no to one space. I'll tally up votes on Friday and update our style guide accordingly. My vote: +1 -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/ mailman/listinfo/juju-dev -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Actions :: UUID vs. Tag on command line
Hi; The current actions spec https://docs.google.com/a/canonical.com/document/d/14W1-QqB1pXZxyZW5QzFFoDwxxeQXBUzgj8IUkLId6cc/edit?usp=sharing indicates that the actions command line should return a UUID as the identifier for an action once it's been en-queued using 'juju do action'. Is there a compelling reason to use UUID's to identify actions, versus using the string representation of the Tag? A UUID would require a command something like: juju status action:9e1e5aa0-5b9d-11e4-8ed6-0800200c9a66 which maybe we could shorten to: juju status action:9e1e5aa0 I would prefer something like: juju status action:mysq/0_a_3 which would be the string representation of the actions Tag. Is there a compelling reason to use UUID? Cheers, -- John Weldon -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Fwd: Actions :: UUID vs. Tag on command line
Forgot to reply-all -- Forwarded message -- From: John Weldon johnweld...@gmail.com Date: Fri, Oct 24, 2014 at 11:19 AM Subject: Re: Actions :: UUID vs. Tag on command line To: Gustavo Niemeyer gustavo.nieme...@canonical.com On Fri, Oct 24, 2014 at 11:14 AM, Gustavo Niemeyer gustavo.nieme...@canonical.com wrote: I doubt this would work. There's no way in the transaction package for you to generate an id and reference that same id in other fields in one go. In other cases that's not an issue, but having a sequence of numbered actions where 10 is applied before 9 would be awkward. Interesting. 1. The sequence is generated in a separate transaction before being used. (state/sequence.go) So I don't think your concern about obtaining and using in one transaction will be an issue. 2. We have not had much discussion around strict ordering of actions being run in the order they were queued. My impression is that two different users interacting with the system at the same time is a bit of an edge case. -- John Weldon -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Actions :: UUID vs. Tag on command line
On Fri, Oct 24, 2014 at 11:23 AM, Gustavo Niemeyer gust...@niemeyer.net wrote: Both of these assumptions are incorrect. Please do not assume there's a single person managing an environment, and the fact the sequence is generated outside of the transaction that adds the action is a proof that actions will be arbitrarily executed rather than in the sequence suggested by the numbers. That's good to know. Ordered execution wasn't addressed in the spec, and we haven't had much discussion about it. I'm not even sure how to enforce ordered execution unless we rely on the creation timestamp. Assuming we have a way to enforce ordered execution, and if that ordering is not using the sequence number that is generated, then does exposing that sequence number just introduce confusion? i.e. are we back to just showing some sort of hash / hex sequence as the id to avoid implying an order by the sequence number? -- John Weldon -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Actions :: UUID vs. Tag on command line
Agreed completely; My take away - 1. Actions en-queued by the same client MUST execute in the order en-queued. 2. Actions en-queued by different clients SHOULD execute in timestamp order? 3. Action IDs should not mislead users by implying sequence that does not exist. 4. ergo Action id's will probably be reflected back to the user in some sort of a manageable hash or hex format -- John Weldon On Fri, Oct 24, 2014 at 11:38 AM, Gustavo Niemeyer gust...@niemeyer.net wrote: On Fri Oct 24 2014 at 4:30:38 PM John Weldon johnweld...@gmail.com wrote: Ordered execution wasn't addressed in the spec, and we haven't had much discussion about it. I'm not even sure how to enforce ordered execution unless we rely on the creation timestamp. Specifications are guidelines. If there are open issues in the specifications, it does not mean that it is okay to do anything in that sense, but rather than either it should be done in the obviously correct way, or that a conversation should be raised if the correct way is not obvious. If someone sends an action, and then sends another action, to me it's clear that the first action should be executed before the second action. If the implementation is not doing that, it should. If two people send two actions concurrently, by definition there's no order implied by their use of the system, and so it's impossible to guarantee which one will be executed first. Assuming we have a way to enforce ordered execution, and if that ordering is not using the sequence number that is generated, then does exposing that sequence number just introduce confusion? How do you feel about postgres action 103 executing before postgres action 102? I personally feel like it's a bug. i.e. are we back to just showing some sort of hash / hex sequence as the id to avoid implying an order by the sequence number? Either option sounds fine to me. I'm only suggesting that if you do use sequence numbers, you're implying a sequence, and people in general are used to being 35 years old only after they've been 34. -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Actions :: UUID vs. Tag on command line
Hmm; makes sense, but this will require some refactoring, because the watcher collects and returns the id's of new actions as an unordered set, as it stands today. I'll start working on this. -- John Weldon On Fri, Oct 24, 2014 at 11:48 AM, Gustavo Niemeyer gust...@niemeyer.net wrote: For 2, it doesn't matter much if the timestamp is taken into account. The server may simply enqueue the action as it receives it and respond back only afterwards. This will guarantee read-your-writes consistency, and thus proper ordering assuming the server does use a queue rather than an unordered set. On Fri Oct 24 2014 at 4:44:03 PM John Weldon johnweld...@gmail.com wrote: Agreed completely; My take away - 1. Actions en-queued by the same client MUST execute in the order en-queued. 2. Actions en-queued by different clients SHOULD execute in timestamp order? 3. Action IDs should not mislead users by implying sequence that does not exist. 4. ergo Action id's will probably be reflected back to the user in some sort of a manageable hash or hex format -- John Weldon On Fri, Oct 24, 2014 at 11:38 AM, Gustavo Niemeyer gust...@niemeyer.net wrote: On Fri Oct 24 2014 at 4:30:38 PM John Weldon johnweld...@gmail.com wrote: Ordered execution wasn't addressed in the spec, and we haven't had much discussion about it. I'm not even sure how to enforce ordered execution unless we rely on the creation timestamp. Specifications are guidelines. If there are open issues in the specifications, it does not mean that it is okay to do anything in that sense, but rather than either it should be done in the obviously correct way, or that a conversation should be raised if the correct way is not obvious. If someone sends an action, and then sends another action, to me it's clear that the first action should be executed before the second action. If the implementation is not doing that, it should. If two people send two actions concurrently, by definition there's no order implied by their use of the system, and so it's impossible to guarantee which one will be executed first. Assuming we have a way to enforce ordered execution, and if that ordering is not using the sequence number that is generated, then does exposing that sequence number just introduce confusion? How do you feel about postgres action 103 executing before postgres action 102? I personally feel like it's a bug. i.e. are we back to just showing some sort of hash / hex sequence as the id to avoid implying an order by the sequence number? Either option sounds fine to me. I'm only suggesting that if you do use sequence numbers, you're implying a sequence, and people in general are used to being 35 years old only after they've been 34. -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Juju Actions - Use Cases
On Wed, Sep 10, 2014 at 9:21 AM, Jeff Pihach jeff.pih...@canonical.com wrote: The interesting part of these interactions vs the ones I've seen mentioned is that they are client side actions. They would open a new tab or the browser to display the page in question vs simply executing some backend script. This does sound interesting - some way of coordinating activity on the client of the API and on the Unit while it's performing the Action. ISTM that features like this, for example OAuth token exchange, have historically required multi-phase steps, where the client initiates a request, the server responds with instructions to complete a third party step, the client performs the step and then submits another request with the resulting token, and maybe some other token to tie the second request to the first one? I'd love some ideas on how to best integrate this into an API for actions. Thanks! -- John Weldon -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Juju Actions - Use Cases
On Tue, Sep 9, 2014 at 11:59 AM, John Weldon johnweld...@gmail.com wrote: If you have any interest or investment in using or publishing Actions for Juju please review and contribute! I'm very thankful for all the discussion so far. I'll try to capture all of the suggestions here in the referenced doc, but feel free to update the doc yourselves too. Cheers -- John Weldon -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Juju Actions - Use Cases
Hi; We're looking for use cases for Juju Actions, mostly to make sure we expose the right API. I'm hoping for a few different use cases from the Juju Web UI folks, but I'd appreciate input from anyone wanting to use Juju Actions in their charms too. I've started a document with some example use cases to prime the pump: please contribute to this document and don't feel constrained to the style or layout I adopted for the examples. If you have any interest or investment in using or publishing Actions for Juju please review and contribute! Google Docs Link https://docs.google.com/document/d/1uYffkkGA1njQ1oego_h8BYBMrlGpmN_lwsnrOZFxE9Q/edit?usp=sharing Thanks -- John Weldon -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: RFC: mongo _id fields in the multi-environment juju server world
On Fri, Jul 4, 2014 at 6:56 AM, Gustavo Niemeyer gust...@niemeyer.net wrote: 1. change the _id field to be a composed field where it is the concatenation of the environment id and the existing id or name field. If we do take this approach, I strongly recommend having the fields that make up the key be available by themselves elsewhere in the document structure. I'd go with this, including your suggestion of splitting the data apart in proper fields. Sounds straightforward and comfortable to deal with. I'd be interested in trying this approach with Actions. We've gone back and forth between encoding units *only* in the _id or *also* in the document. Both have pro's and con's, but it seems to me that a composite _id would address most of the con's on each approach. I'm also interested in figuring out how the watchers will work in this approach. The Actions watcher is a StringsWatcher, and the .Changes() are []string I'm assuming that will have to become a more specialised watcher where .Changes() returns a list of objects representing the composite key? Also how the watcher detects relevant events might have to be adjusted somewhat. -- John Weldon -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: not rebasing after PR?
On Thu, Jun 5, 2014 at 8:22 AM, Nate Finch nate.fi...@canonical.com wrote: commit several times to your feature branch. rebase into a single commit I don't see any problem with rebasing commits that you've already pushed to your github personal fork, but that *could* be considered changing public history. That's how I have been working already, but I'd like to know if that is a problem for some reason. -- John Weldon -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Actions
We're working on Actions ... it's going in fits and starts. We've got a lot to learn (and appreciate all the time and attention from all of you experienced juju devs) We were thinking maybe by the middle of June, but we've been over-optimistic in the past. Hopefully soon! irc: jcw4, bodie_, and sometimes bits3rpent :) -- John Weldon On Wed, May 14, 2014 at 4:13 PM, Mike Sam mikesam...@gmail.com wrote: I know juju run is out but I am wondering when we should expect Actions that do not require ssh access to the machines? Also, will actions allow arbitrary (defined at runtime) hook code or show the code be part of the charm already? Thanks, Mike -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
