Re: Should 'juju status' always include a timestamp
I'm also +1 on a timestamp. On Wed, Apr 18, 2018 at 8:22 AM, Jason Hobbs wrote: > Yes, a timestamp is always appropriate. > > On Wed, Apr 18, 2018 at 3:12 AM, Merlijn Sebrechts > wrote: > > +1 > > > > Op wo 18 apr. 2018 om 09:19 schreef John A Meinel > > : > >> > >> I was just going over a list of pastes from James Beedy on this bug: > >> https://bugs.launchpad.net/juju/+bug/1763963 > >> > >> And I realized that something that would be really nice is to just have > >> some sort of timestamp so you could see how much time elapsed between > each > >> "juju status". And given the use of 'juju status' it seems like > something > >> that would generally be useful. (sharing a snapshot with someone else > would > >> always be done with the context of *when* you ran that status, so you > could > >> correlate it with other information like load metrics, etc.) > >> > >> Thoughts? It seems easy enough to add, but do people feel that it > >> does/doesn't belong? > >> > >> John > >> =:-> > >> > >> -- > >> Juju-dev mailing list > >> Juju-dev@lists.ubuntu.com > >> Modify settings or unsubscribe at: > >> https://lists.ubuntu.com/mailman/listinfo/juju-dev > > > > > > -- > > Juju-dev mailing list > > Juju-dev@lists.ubuntu.com > > Modify settings or unsubscribe at: > > https://lists.ubuntu.com/mailman/listinfo/juju-dev > > > > -- > Juju-dev mailing list > Juju-dev@lists.ubuntu.com > Modify settings or unsubscribe at: https://lists.ubuntu.com/ > mailman/listinfo/juju-dev > -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Statistics and metrics about Charm, layer and interface usage
I meant to follow up with this last week but lost track. In addition to the raw API for getting charm content, you can also just do `charm pull $CHARM_ID` to get the archive zip, in case that's easier. For charm stats, the API docs that Rick linked to includes the "stats" endpoint. You can see some example usage of that in https://github.com/juju-solutions/k8s-kpi-scripts/blob/master/src/files/thirdparty/k8s-charm-cs-stats For layer and interfaces, you can use the GitHub API to query how many layers and interfaces are available (ex. https://github.com/juju-solutions/charmbuild-kpi-import/blob/master/scripts/github) but unfortunately there's no easy way at this time to tell how many charms a given layer is used in. On Fri, Jan 26, 2018 at 3:25 AM, Merlijn Sebrechts < merlijn.sebrec...@gmail.com> wrote: > Awesome! Thanks guys! > > 2018-01-25 17:38 GMT+01:00 Rick Harding : > >> https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md is the >> docs for the API to the charmstore itself. As Adam notes, you can pull down >> any file and there's manifest call that lists out the files in the charm. >> From there you could probably check if the charm has a layers.yaml and if >> so fetch that file, parse it, etc. >> >> https://github.com/juju/charmstore/blob/v5-unstable/docs/ >> API.md#get-idmetamanifest >> >> On Thu, Jan 25, 2018 at 11:22 AM Adam Collard >> wrote: >> >>> Hi Merlijn, >>> >>> >>> On Thu, 25 Jan 2018 at 16:17 Merlijn Sebrechts < >>> merlijn.sebrec...@gmail.com> wrote: >>> Hi all I'm writing a Juju-related paper and I'd like to get statistics on Charm, layer and interface usage. Are these publicly available? Related: is there a documented API to get the code of the charms that are available in the charm store? >>> >>> GET https://api.jujucharms.com/charmstore/v5/$MY_CHARM_NAME/archive/ >>> will give you a .zip >>> and >>> >>> GET https://api.jujucharms.com/charmstore/v5/$MY_CHARM_NAME/arch >>> ive/$PATH/$TO/$FILE >>> will give you the 'raw' contents. >>> >>> e.g. curl https://api.jujucharms.com/charmstore/v5/postgresql/archive/ >>> hooks/install >>> >>> YMMV, >>> >>> Adam >>> -- >>> Juju-dev mailing list >>> Juju-dev@lists.ubuntu.com >>> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailm >>> an/listinfo/juju-dev >>> >> > > -- > Juju-dev mailing list > Juju-dev@lists.ubuntu.com > Modify settings or unsubscribe at: https://lists.ubuntu.com/ > mailman/listinfo/juju-dev > > -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Charm layer index now on GitHub
Keep in mind that the charm-build update is really more housekeeping to allow us to retire the old interfaces.juju.solutions domain entirely. All charm builds should now be using the new repo. But of course testing of that change is more than welcome. On Fri, Sep 8, 2017 at 3:00 PM, Marco Ceppi wrote: > This seems to have landed in candidate already, for those wanting to test > on something other than edge. > > sudo snap refresh charm --channel candidate > > On Fri, Sep 8, 2017 at 2:59 PM Tim Van Steenburgh < > tim.van.steenbu...@canonical.com> wrote: > >> Fantastic, this is a great change - well done! >> >> On Fri, Sep 8, 2017 at 2:39 PM, Cory Johns >> wrote: >> >>> Greetings, >>> >>> Today we migrated the index of base and interface layers used to build >>> charms over to the GitHub repository https://github.com/juju/layer-index. >>> Hosting the index in GitHub provides better discoverability for layers, a >>> better workflow for contributing layers, including more accountability for >>> changes, and both more flexibility and more visibility with community >>> contributions. It also reduces our maintenance overhead and removes a >>> point of failure. >>> >>> The change should be seamless to the build process, and the existing >>> http://interfaces.juju.solutions/ site now redirects to the new repo. >>> An updated charm-build command is now in the edge channel which points >>> directly to the new index, and the old site will eventually be taken down >>> once that reaches the stable channel and some time has passed. >>> >>> I’m now happy to say, PRs welcome! >>> >>> -- >>> Juju-dev mailing list >>> Juju-dev@lists.ubuntu.com >>> Modify settings or unsubscribe at: https://lists.ubuntu.com/ >>> mailman/listinfo/juju-dev >>> >>> >> -- >> Juju-dev mailing list >> Juju-dev@lists.ubuntu.com >> Modify settings or unsubscribe at: https://lists.ubuntu.com/ >> mailman/listinfo/juju-dev >> > -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Charm snap is now strictly confined
Greetings, I just wanted to make a quick announcement that the charm snap is now strictly confined on the stable channel (rev 17). This fixes issues like https://github.com/juju/charm-tools/issues/339 and https://github.com/juju/charm-tools/issues/319 and prevents similar issues from cropping up in the future. In general, this change should be pretty much transparent, with the one caveat being that you can only build charms from under your HOME directory. -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Charm layer index now on GitHub
Greetings, Today we migrated the index of base and interface layers used to build charms over to the GitHub repository https://github.com/juju/layer-index. Hosting the index in GitHub provides better discoverability for layers, a better workflow for contributing layers, including more accountability for changes, and both more flexibility and more visibility with community contributions. It also reduces our maintenance overhead and removes a point of failure. The change should be seamless to the build process, and the existing http://interfaces.juju.solutions/ site now redirects to the new repo. An updated charm-build command is now in the edge channel which points directly to the new index, and the old site will eventually be taken down once that reaches the stable channel and some time has passed. I’m now happy to say, PRs welcome! -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
New release of python-libjuju
The 0.6.0 release of the Python bindings for the Juju API is now available on PyPI. The docs have moved to ReadTheDocs, due to PythonHosted being deprecated, and the change log can be found at: https://pythonlibjuju.readthedocs.io/en/latest/changelog.html Here is a synopsis of the recent releases: 0.6.0 --- Thu June 29 2017 * Implement scp functionality (#149) * Add Unit.public_address property (#153) * Adds support for getting/setting config on a model (#152) 0.5.3 --- Thu June 22 2017 * Improve handling of closed connections (#148) * Configurable and larger max message size (#146) 0.5.2 --- Wed June 14 2017 * Fix deploying non-stable channels and explicit revs (#144) -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Maximum AllWatcher frame size?
So, I'm not sure if this resolves the issue. The python websockets library supports fragmented messages but I think that is separate from the maximum message size that it's enforcing. I think that maximum is for the final payload, which would be the entire AllWatcher JSON response. If the controller doesn't break that in to multiple AllWatcher updates, then I think we'll still hit the maximum message size. I can increase the max size pretty easily, or even remove it entirely, as mentioned in the issue, but I'm a little bit leery of having no upper bound at all. Do we know if there's a reasonable upper bound that even large models would fall within for the initial AllWatcher response? On Wed, Jun 14, 2017 at 12:08 PM, Tim Penhey wrote: > It is configured in juju > > > // Use a 64k frame size for the websockets while we need to deal > // with x/net/websocket connections that don't deal with recieving > // fragmented messages. > const websocketFrameSize = 65536 > > > On 14/06/17 12:00, Cory Johns wrote: > > Do we know what size the gorilla/websocket library uses for automatic > > chunking? > > > > On Wed, Jun 14, 2017 at 11:35 AM, John Meinel > <mailto:j...@arbash-meinel.com>> wrote: > > > > In 2.1 we did not chunk, in 2.2 we switch to gorilla/websocket which > > does support chunking into frames. I don't think we do any internal > > "well that would be too much information so we wont send it all". > > > > John > > =:-> > > > > > > On Wed, Jun 14, 2017 at 7:11 PM, Cory Johns > > mailto:cory.jo...@canonical.com>> wrote: > > > > https://github.com/juju/python-libjuju/issues/136 > > <https://github.com/juju/python-libjuju/issues/136> raises the > > issue that, for large models, the initial AllWatcher response > > frame can be large enough that it overruns the default maximum > > frame size of the websocket library (1MB). We can increase this > > limit fairly easily, but I wanted to find out whether there is > > any maximum size from the Juju size and if large enough frames > > could get chunked into multiple frames? > > > > -- > > Juju-dev mailing list > > Juju-dev@lists.ubuntu.com <mailto:Juju-dev@lists.ubuntu.com> > > Modify settings or unsubscribe at: > > https://lists.ubuntu.com/mailman/listinfo/juju-dev > > <https://lists.ubuntu.com/mailman/listinfo/juju-dev> > > > > > > > > > > > -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Maximum AllWatcher frame size?
Do we know what size the gorilla/websocket library uses for automatic chunking? On Wed, Jun 14, 2017 at 11:35 AM, John Meinel wrote: > In 2.1 we did not chunk, in 2.2 we switch to gorilla/websocket which does > support chunking into frames. I don't think we do any internal "well that > would be too much information so we wont send it all". > > John > =:-> > > > On Wed, Jun 14, 2017 at 7:11 PM, Cory Johns > wrote: > >> https://github.com/juju/python-libjuju/issues/136 raises the issue that, >> for large models, the initial AllWatcher response frame can be large enough >> that it overruns the default maximum frame size of the websocket library >> (1MB). We can increase this limit fairly easily, but I wanted to find out >> whether there is any maximum size from the Juju size and if large enough >> frames could get chunked into multiple frames? >> >> -- >> Juju-dev mailing list >> Juju-dev@lists.ubuntu.com >> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailm >> an/listinfo/juju-dev >> >> > -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Maximum AllWatcher frame size?
https://github.com/juju/python-libjuju/issues/136 raises the issue that, for large models, the initial AllWatcher response frame can be large enough that it overruns the default maximum frame size of the websocket library (1MB). We can increase this limit fairly easily, but I wanted to find out whether there is any maximum size from the Juju size and if large enough frames could get chunked into multiple frames? -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Juju 2.2-rc2 has been released
Simon, That was my mistake, I forgot to click release on that. conjure-up-rc2-20170612 is now available on the candidate channel. On Fri, Jun 9, 2017 at 8:06 PM, Simon Kollberg wrote: > conjure-up still seem to be on rc1 > > % sudo snap install conjure-up --classic --candidate > conjure-up (candidate) 2.2-rc1-20170606.0106 from 'canonical' installed > % juju --version > 2.2-rc1-xenial-amd64 > % conjure-up --version > conjure-up 2.2-rc1 > % sudo snap remove conjure-up > conjure-up removed > % sudo snap install juju --classic --candidate > juju (candidate) 2.2-rc2 from 'canonical' installed > % juju --version > 2.2-rc2-xenial-amd64 > > On 9 June 2017 at 13:25, Chris Lee wrote: > >> # Juju 2.2-rc2 Release Notes >> >> >> >> We are delighted to announce the release of Juju and conjure-up 2.2-rc2! >> In this release, Juju greatly improves memory and storage consumption, >> works on KVM containers, and improves network modelling. conjure-up now >> supports Juju as a Service (JAAS), provides a MacOS client, and adds >> support for repeatable spell deployments. >> >> >> >> The best way to get your hands on this release of Juju and conjure-up is >> to install them via snap packages (see https://snapcraft.io/ for more >> info on snaps). >> >> >> >> snap install juju --classic --candidate >> >> snap install conjure-up --classic --candidate >> >> >> >> Other packages are available for a variety of platforms. Please see the >> online documentation at https://jujucharms.com/docs/de >> vel/reference-releases#development >> >> >> >> Please note that if you are upgrading an existing controller, please make >> sure there is at least 6G of free disk space. The upgrade step for the logs >> can take a while, in the vicinity of 10 or more minutes if the current logs >> collection is at its maximum size. >> >> >> >> Since 2.2-rc1 >> >> >> ## New and Improved >> >> -- >> >> >> >> Better support credential management in the Azure provider >> >> * support autoload-credentials and juju add-credential in the azure >> provider when Azure CLI is installed. >> >> (this removes the requirement that the user discover their subscription >> ID before creating credentials) >> >> >> >> Rate limit login and connection requests to the controller(s) on busy >> systems. >> >> >> >> ## Fixes >> >> -- >> >> >> >> Fix issue where status history logs were not pruned: >> >> https://bugs.launchpad.net/juju/+bug/1696491 >> >> >> -- >> Juju mailing list >> j...@lists.ubuntu.com >> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailm >> an/listinfo/juju >> >> > > -- > Juju mailing list > j...@lists.ubuntu.com > Modify settings or unsubscribe at: https://lists.ubuntu.com/ > mailman/listinfo/juju > > -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Leadership Election Tools
I think that's another nail in the coffin for @hook. You could work around
it by having your @hook('upgrade-charm') handler unconditionally set a
state (e.g., 'upgrade-charm'), and then have the state handler do the
gating on leadership. This would also allow you to use the decorators for
gating on leadership.is_leader, rather than using the is_leader() function.
I definitely want to move toward having the reactive framework itself
manage states like upgrade-charm, and {relation_name}.joined so that we can
remove all uses of @hook.
On Wed, Dec 14, 2016 at 6:00 AM, Stuart Bishop
wrote:
>
>
> On 14 December 2016 at 00:39, Matthew Williams <
> matthew.willi...@canonical.com> wrote:
>
>> Hey Folks,
>>
>> Let's say I'm a charm author that wants to test leadership election in my
>> charm. Are there any tools available that will let me force leadership
>> election in juju so that I can test how my charm handles it? I was looking
>> at the docs here: https://jujucharms.com/docs/stable/developer-leadership
>> but couldn't see anything
>>
>
> I don't think there is any supported way of doing this.
>
> If you don't mind an unsupported hack though, use 'juju ssh' to shut down
> the unit's jujud, wait 30 seconds for the lease to expire, and you should
> have a new leader. 'juju ssh' again to restart the jujud, 'juju wait' for
> the hooks to clear, and failover is done. 'juju run' will hang if you use
> it to shutdown jujud, so don't do that.
>
> juju ssh ubuntu/0 'sudo systemctl stop jujud-unit-ubuntu-0.service'
> sleep 30
> juju ssh ubuntu/0 'sudo systemctl stop jujud-unit-ubuntu-0.service'
> juju wait
>
> Ideally, you may be able to structure things so that it doesn't matter
> which unit is leader. If all state relating to leadership decisions is
> stored in the leadership settings, and if you avoid using @hook, then it
> doesn't matter which unit makes the decisions. Worst case is that *no* unit
> is leader when hooks are run, and decisions get deferred until
> leader-elected runs.
>
> (Interesting race condition for the day: It is possible for all units in a
> service to run their upgrade-charm hook and for none of them to be leader
> at the time, so @hook('upgrade-charm') code guarded by is-leader may never
> run. And reactive handlers have no concept of priority and might kick in
> rather late for upgrade steps, requiring more creative use of reactive
> states to guard 'new' code from running too soon. Not specific to
> upgrade-charm hooks either, so avoid using @hook and leadership together)
>
>
> --
> Stuart Bishop
>
> --
> Juju-dev mailing list
> Juju-dev@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/juju-dev
>
>
--
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: A (Very) Minimal Charm
Marco, What is the issue you mentioned with using snaps where you mentioned needing an "unconfined classic snap"? On Thu, Dec 1, 2016 at 1:13 PM, Marco Ceppi wrote: > On Thu, Dec 1, 2016 at 12:56 PM Casey Marshall < > casey.marsh...@canonical.com> wrote: > > On Thu, Dec 1, 2016 at 6:53 AM, Marco Ceppi > wrote: > > On Thu, Dec 1, 2016 at 5:00 AM Adam Collard > wrote: > > On Thu, 1 Dec 2016 at 04:02 Nate Finch wrote: > > On IRC, someone was lamenting the fact that the Ubuntu charm takes longer > to deploy now, because it has been updated to exercise more of Juju's > features. My response was - just make a minimal charm, it's easy. And > then of course, I had to figure out how minimal you can get. Here it is: > > It's just a directory with a metadata.yaml in it with these contents: > > name: min > summary: nope > description: nope > series: > - xenial > > (obviously you can set the series to whatever you want) > No other files or directories are needed. > > > This is neat, but doesn't detract from the bloat in the ubuntu charm. > > > I'm happy to work though changes to the Ubuntu charm to decrease "bloat". > > > IMHO the bloat in the ubuntu charm isn't from support for Juju features, > but the switch to reactive plus conflicts in layer-base wanting to a) > support lots of toolchains to allow layers above it to be slimmer and b) be > a suitable base for "just deploy me" ubuntu. > > > But it is to support the reactive framework, where we utilize newer Juju > features, like status and application-version to make the charm rich > despite it's minimal goal set. Honestly, a handful of cached wheelhouses > and some apt packages don't strike me as bloat, but I do want to make sure > the Ubuntu charm works for those using it. So, > > > I think a minimal wheelhouse to provide a consistent charm hook runtime is > very reasonable and definitely not the problem here. > > There are too many packages that get installed by default with the > reactive framework that most charms don't need. When I deploy the ubuntu > charm (but this applies to any charm built with reactive and layer:basic), > I also get: > > 2016-12-01 17:45:47 INFO install The following NEW packages will be > installed: > 2016-12-01 17:45:47 INFO install binutils build-essential cpp cpp-5 > dpkg-dev fakeroot g++ g++-5 gc > c gcc-5 > 2016-12-01 17:45:47 INFO install libalgorithm-diff-perl > libalgorithm-diff-xs-perl libalgorithm-mer > ge-perl > 2016-12-01 17:45:47 INFO install libasan2 libatomic1 libc-dev-bin > libc6-dev libcc1-0 libcilkrts5 l > ibdpkg-perl > 2016-12-01 17:45:47 INFO install libexpat1-dev libfakeroot > libfile-fcntllock-perl libgcc-5-dev libgomp1 > 2016-12-01 17:45:47 INFO install libisl15 libitm1 liblsan0 libmpc3 > libmpx0 libpython3-dev libpython3.5-dev > 2016-12-01 17:45:47 INFO install libquadmath0 libstdc++-5-dev libtsan0 > libubsan0 linux-libc-dev make > 2016-12-01 17:45:47 INFO install manpages-dev python-pip-whl python3-dev > python3-pip python3-setuptools > 2016-12-01 17:45:47 INFO install python3-wheel python3.5-dev > > None of my charms need build-essential or a g++ compiler, that's a lot of > unnecessary dependencies! Can we get rid of most of these? Would installing > the bare minimum with --no-install-recommends help? > > > This comes up a bit, and I'm really eager to figure this out. Allow me to > explain the catch-22. It's name is pyyaml. > > So the wheelhouse, by default, is 3.8M in size, this is the stock > wheelhouse we vendor in: > > 312K charmhelpers-0.10.0.tar.gz > 21K charms.reactive-0.4.5.tar.gz > 349K Jinja2-2.8.tar.gz > 14K MarkupSafe-0.23.tar.gz > 1.7M netaddr-0.7.18.tar.gz > 1.1M pip-8.1.2.tar.gz > 19K pyaml-16.11.4.tar.gz > 248K PyYAML-3.12.tar.gz > 29K six-1.10.0.tar.gz > 13K Tempita-0.5.2.tar.gz > > The problem child is PyYAML, which is a compiled cpyhton module, which > requires build-essential. The latest version is 3.12, trusty is 3.10, > xenial 3.11, and zesty (finally) 3.12 http://packages.ubuntu. > com/search?keywords=python-yaml, CentOS 6 & 7 are 3.10 > > So, the easy path here is to just make sure charms.reactive works with > PyYAML 3.10 and do a `--no-install-recommends` but that's half the problem. > There will inevitably be python modules that require build-essential to > compile on install. > > We can't cache the compiled wheelhouse because of architectures (same way > resources complicate this) so we must compile at deploy time. > > One path forward, after dropping PyYAML 3.12 (if feasible), would be to > see if we can detect when a python module needs to be compiled and setting > a flag in the rendered charm to install the build-essentials, etc. > > I'll file issues and spend some time seeing if we can actually detect when > you need to compile a wheelhouse vs a straight python module that and > lowering the requirement for PyYAML (using packaged version instead) will > probably remove a lot of the reactive bootstrap time. > > Marco > > -- > Juju-dev mailing list > J
Re: New helper library for unit-testing charms
Free, This looks fantastic! Can I request that you cross-post this over to the main Juju mailing list, as it would be very helpful for the community as a whole? On Tue, Oct 11, 2016 at 4:57 AM, Free Ekanayaka < free.ekanay...@canonical.com> wrote: > Hello, > > I published a couple of new Python packages aimed at making unit tests for > charms easier and nicer: > > https://github.com/freeekanayaka/charm-test > https://github.com/freeekanayaka/systemfixtures > > They implement the classic approach [0] of replacing the "boundaries" of a > charm (hook tools, processes, filesystem, network, etc) with realistic > fakes, so your unit tests run actual code as much as possible and you don't > have to mock internal APIs that directly or indirectly interact with the > boundaries (often leading to mocking explosion). > > Besides the package documentation, you might want to have a look to the > jenkins charm reactive rewrite branch [1] to see some real-world use. > > If there's interest around this, we could move the code under the juju > GitHub tree, or even merge it to the charm.unit package [2]. > > Cheers, > > Free > > [0] http://martinfowler.com/articles/mocksArentStubs.html > [1] https://github.com/freeekanayaka/layer-jenkins > [2] https://github.com/juju-solutions/charms.unit > > -- > Juju-dev mailing list > Juju-dev@lists.ubuntu.com > Modify settings or unsubscribe at: https://lists.ubuntu.com/ > mailman/listinfo/juju-dev > > -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: New feature for 1.26 (master), $(JUJU_HOME)/aliases
You can accomplish most of this with the existing plugin system. You can't override existing commands, but you can easily create thin wrappers around them with your desired default args, and given the discussion around --no-aliases, it seems like this is actually a benefit. And plugins provide an easy way to add more intelligent handling of args, and also come with zero additional dev burden on core. On Tue, Oct 27, 2015 at 3:24 AM, John Meinel wrote: > The one thing I don't think bash aliases give you is proper handling of > nested subcommands. Can you easily do "juju st" in a bash alias? or does it > have to become "juju-st" ? > I think the caveats around people aliasing their way out of clumsiness in > the tool is actually something to pay close attention to. It is very easy > to customize a tool around making it work great the way you want, but the > default experience for someone else is poor. (things like "juju status > --format=tabular" is one of those key places where we need to think > carefully about a great experience vs compatibility) > > It *is* a way to prototype "what if the commands acted this way" for us, > and to let users customize how they want to use the tool. But we need to be > rather proactive about turning our findings into actual changes to the > default behavior of the tool. > > Thinking about "--no-aliases" actually runs into the problem that it isn't > backwards compatible, either. You can't supply --no-aliases to old versions > of Juju, and you have to supply it to new ones in scripts. It feels like > ENV vars work a bit better here, as a script can set whatever ENV vars at > the beginning and then leave the rest of the script alone. (I'm pretty sure > that's why JUJU_CLI_VERSION=2 or whatever the exact syntax is, ended up as > an ENV var.) > > John > =:-> > > On Tue, Oct 27, 2015 at 7:43 AM, Nate Finch > wrote: > >> I don't think cross platform support of the config file format justifies >> the non-zero maintenance burden of the feature. Nor the non-zero >> maintenance burden of all the features that will inevitably follow after >> it, like the new --no-aliases flag, which is now cognitive overhead on >> every single command and every single script file. >> >> Also, I think Rick has it right - aliases mean users end up with >> different experiences of the product. This happened to me all the time with >> bzr plugins - first Martin would have to figure out what plugins I was >> using and what format my repo was in, and then he could help me figure out >> what I'd screwed up. ;) And people using aliases often forget that the >> command they type all the time is not actually built into the product. I >> can't tell you how often I've had someone tell me to 'git co ' >> (which is a common - but not universal - alias for git checkout). >> >> If you really want to use aliases, bash aliases work nearly identically, >> AFAICT, but with truly zero maintenance burden on core devs. By adding >> alias support to Juju, you're not only condoning aliases, you're >> encouraging their use, and I think that's a bad idea. If the UX of some of >> our commands suck, we should fix that. If our backwards compatibility >> policy is preventing us from fixing that, then at least we know the true >> cost of the policy - since most users in the wild are *not* going to be >> using any aliases. We should be using Juju in the same way as our >> less-experienced users, since all of our users will have to go through that >> period. (I'll ignore for the moment that quickstart has the exact same >> problem and is actually way more disruptive to the UX). >> >> -Nate >> >> >> On Mon, Oct 26, 2015 at 9:26 PM Tim Penhey >> wrote: >> >>> Hi Wayne, >>> >>> While I can see your point of view, I do disagree. >>> >>> While it may be of no benefit to you, it doesn't mean that others will >>> not find it useful. >>> >>> I believe it is useful to have one format to specify aliases, one that >>> doesn't depend on the shell that you use. >>> >>> The overhead in complexity isn't really that much, and with the >>> --no-alias option which is coming, it still provides a way to ensure >>> vanilla behaviour. >>> >>> >>> Tim >>> >>> On 27/10/15 04:03, Wayne Witzel wrote: >>> > After looking at this and the code more, I find myself very against >>> this >>> > feature. This adds code to core that is performing a tasks that most >>> > people already know is handled by their shell. Even our Windows users >>> > can install PowerShell and have aliases for commands. This adds no >>> > benefit to core, it adds more complexity, and attempts to perform a >>> task >>> > that is already well handled by the users shell. If anything, this itch >>> > could be scratched by adding some contrib documentation about some of >>> > your favourite and/or most used aliases. >>> > >>> > On Mon, Oct 26, 2015 at 10:51 AM, Wayne Witzel >>> > mailto:wayne.wit...@canonical.com>> >>> wrote: >>> > >>> > What is the advantage of this over using a st
Re: Juju Resources (a tool / library)
Gustavo, Can you be more specific on how it differs from the goals of resources streams? As I mentioned in my first email, I asked around to try to get specific information about the proposed feature and wasn't able to get any concrete answers or documentation. So I created this based on what I remembered from the discussions I'd heard (admittedly not much) and what I needed in the charms I was working on. I fully intend for this library to be subsumed / obviated by core as the feature develops, and I tried to make that clear in the library README and documentation. I also intend to update the interface to match the feature as closely as possible as the proposal becomes more concrete. On Wed, Feb 11, 2015 at 2:33 PM, Gustavo Niemeyer wrote: > Hi Cory, > > While it's fine and welcome to have such test bed features, it feels > like the proposal and implementation have quite different goals from > the actual resources feature we've been talking about for a while, so > as a very early suggestion and request, I would strongly recommend > renaming the feature to avoid creating ambiguity with the feature that > we intend juju to have in the long run. Having two resource > implementations and taking over important namespaces such as > resources.yaml might introduce unnecessary confusion down the road. > Instead, the project might have a nice non-generic name, and its > configuration file could also be named after it. > > > On Wed, Feb 11, 2015 at 4:17 PM, Cory Johns wrote: >> Per request, the documentation is now also available on >> ReadTheDocs.org: http://jujuresources.readthedocs.org/ >> >> On Wed, Feb 11, 2015 at 11:25 AM, Cory Johns >> wrote: >>> Hi all, >>> >>> (cross-posting to juju & juju-dev) >>> >>> I've created a tool / library for organizing and managing resources >>> (binary blobs, tarballs, Python packages, and, eventually, apt >>> packages) required by a charm. The idea is to be an interim tool, and >>> a test-bed for the resource features that have been discussed for the >>> future in Juju core. >>> >>> It is available on GitHub [1] and PyPI [2], and the full documentation >>> is on PythonHosted.org [3]. >>> >>> The goals of this project are: >>> >>> * Organize all external resource dependencies into a single >>> resources.yaml file >>> * Provide easy, consistent interfaces to manage, install, and mirror >>> resources >>> * CLI and Python bindings >>> * Enforce best practices, such as cryptographic hash validation >>> >>> I asked around to see if there was an existing proposal for a >>> resources.yaml file format, but couldn't find one. If someone is >>> aware of an existing spec / proposal, I'd prefer to match that as much >>> as possible. >>> >>> The current version is fully functional, and is currently being used >>> in the framework refactor of the Apache Hadoop charms (e.g., [4]). >>> >>> Note that I created this separately from Charm Helpers primarily >>> because I wanted to use it to bootstrap CH, but this also makes it >>> easier to use in Bash charms. >>> >>> My next step is to add apt-get support, but that will requiring >>> cleaning up the mirror server (possibly converting it to use squid, >>> but I may want to keep it self-contained), and learning a bit more >>> about how the apt proxy settings work. Advice here is appreciated. >>> >>> >>> [1] https://github.com/juju-solutions/jujuresources >>> [2] https://pypi.python.org/pypi/jujuresources >>> [3] http://pythonhosted.org/jujuresources/ >>> [4] >>> https://code.launchpad.net/~bigdata-dev/charms/trusty/apache-hadoop-hdfs-master/trunk >> >> -- >> Juju-dev mailing list >> Juju-dev@lists.ubuntu.com >> Modify settings or unsubscribe at: >> https://lists.ubuntu.com/mailman/listinfo/juju-dev > > > > -- > > gustavo @ http://niemeyer.net -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Juju Resources (a tool / library)
Per request, the documentation is now also available on ReadTheDocs.org: http://jujuresources.readthedocs.org/ On Wed, Feb 11, 2015 at 11:25 AM, Cory Johns wrote: > Hi all, > > (cross-posting to juju & juju-dev) > > I've created a tool / library for organizing and managing resources > (binary blobs, tarballs, Python packages, and, eventually, apt > packages) required by a charm. The idea is to be an interim tool, and > a test-bed for the resource features that have been discussed for the > future in Juju core. > > It is available on GitHub [1] and PyPI [2], and the full documentation > is on PythonHosted.org [3]. > > The goals of this project are: > > * Organize all external resource dependencies into a single > resources.yaml file > * Provide easy, consistent interfaces to manage, install, and mirror > resources > * CLI and Python bindings > * Enforce best practices, such as cryptographic hash validation > > I asked around to see if there was an existing proposal for a > resources.yaml file format, but couldn't find one. If someone is > aware of an existing spec / proposal, I'd prefer to match that as much > as possible. > > The current version is fully functional, and is currently being used > in the framework refactor of the Apache Hadoop charms (e.g., [4]). > > Note that I created this separately from Charm Helpers primarily > because I wanted to use it to bootstrap CH, but this also makes it > easier to use in Bash charms. > > My next step is to add apt-get support, but that will requiring > cleaning up the mirror server (possibly converting it to use squid, > but I may want to keep it self-contained), and learning a bit more > about how the apt proxy settings work. Advice here is appreciated. > > > [1] https://github.com/juju-solutions/jujuresources > [2] https://pypi.python.org/pypi/jujuresources > [3] http://pythonhosted.org/jujuresources/ > [4] > https://code.launchpad.net/~bigdata-dev/charms/trusty/apache-hadoop-hdfs-master/trunk -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Juju Resources (a tool / library)
Hi all, (cross-posting to juju & juju-dev) I've created a tool / library for organizing and managing resources (binary blobs, tarballs, Python packages, and, eventually, apt packages) required by a charm. The idea is to be an interim tool, and a test-bed for the resource features that have been discussed for the future in Juju core. It is available on GitHub [1] and PyPI [2], and the full documentation is on PythonHosted.org [3]. The goals of this project are: * Organize all external resource dependencies into a single resources.yaml file * Provide easy, consistent interfaces to manage, install, and mirror resources * CLI and Python bindings * Enforce best practices, such as cryptographic hash validation I asked around to see if there was an existing proposal for a resources.yaml file format, but couldn't find one. If someone is aware of an existing spec / proposal, I'd prefer to match that as much as possible. The current version is fully functional, and is currently being used in the framework refactor of the Apache Hadoop charms (e.g., [4]). Note that I created this separately from Charm Helpers primarily because I wanted to use it to bootstrap CH, but this also makes it easier to use in Bash charms. My next step is to add apt-get support, but that will requiring cleaning up the mirror server (possibly converting it to use squid, but I may want to keep it self-contained), and learning a bit more about how the apt proxy settings work. Advice here is appreciated. [1] https://github.com/juju-solutions/jujuresources [2] https://pypi.python.org/pypi/jujuresources [3] http://pythonhosted.org/jujuresources/ [4] https://code.launchpad.net/~bigdata-dev/charms/trusty/apache-hadoop-hdfs-master/trunk -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Bundles proposal
On Thu, Jun 19, 2014 at 6:52 PM, Richard Harding wrote: > On Thu, 19 Jun 2014, Benjamin Saller wrote: > Haven't we talked about this type of content as a different data type > though than a bundle? Is this something we should work towards at this > time? I think that is what Ben is talking about, that as we move bundles into the core as first-order entities, that is the correct time to address expanding their scope to include the additional functionality we have realized they should cover. >> I also understand that in some world views a bundle would represent a >> supportable collection of services and this might imply the topology is >> fixed but any complex system that evolves is likely to have some topology >> mutation. While we can manage this without first class support it would be >> difficult to expect the GUI to the the 'right' thing for our project unless >> bundles include this idea. > > I'm not sure I completely follow here. The idea is to get a model for a > topology definition. In the GUI we can allow users to deploy the toplogy in > 'ghost' form and then mutate it before submitting it to the environment to > construct it. If we move the idea of the uncommitted state down into juju > core then the same idea could be possible via cli invocation of a bundle. > I'm not sure why bundles have to include the idea of mutation itself. I'm > nervous about things that make bundles not able to be reused as much as > possible because they cannot stand alone. What we've found that we need from a "stack" (ere bundle) is the ability to encapsulate the intelligence to handle upgrades that include modifications to the topology, providing logic to perform the transition on an existing system, while retaining the overall identity of the bundle. As a concrete example, with CF we are in the position that we expect an upgrade from one CF release to the next to potentially include splitting an existing service into two distinct services, or merging two existing services into a single new service. We can sometimes handle the former case by simply having the charm be responsible for both new services, but that isn't always ideal, and it still doesn't cover the second case. -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
