Re: OS X VMS on JAAS

[Jay Wren]

I do not understand how this works. Could someone with knowledge of how
jujud on a  controller communicates with jujud agents on units describe how
that is done?

My limited understanding must be wrong give that James has this working.

This is what I thought:

On most cloud providers: add-machine instructs the cloud provider to start
a new instance and the cloud-config passed to cloud-init includes how to
download jujud agent and run it and configure it with public key trust of
the juju controller.

On manually added machine: same thing only instead of cloud-init and
cloud-config an ssh connection is used to perform the same commands.

I had thought the juju controller was initiating the ssh-connection to the
address given in the add-machine command and that a non-internet routable
address would simply not work as the controller cannot open any TCP
connection to it. This is where my understanding stops.

Please, anyone, describe how this works?
--
Jay


On Fri, Jun 2, 2017 at 9:42 AM, James Beedy  wrote:

> I think the primary advantage being less clutter to the end user. The
> difference between the end user have to bootstrap and control things from
> inside the vm vs from their host. For some reason this small change made
> some of my users who were previously not really catching on, far more apt
> to jump in. I personally like it because these little vms go further when
> they don't have the controller on them as well. @jameinel totally, possibly
> I'll add the bridge bits in place of the lxd-proxy in that write up, or
> possibly in another.
>
> ~James
>
> On Jun 2, 2017, at 12:56 AM, John Meinel  wrote:
>
> Interesting. I wouldn't have thought to use a manually added machine to
> use JAAS to deploy applications to your local virtualbox. Is there a reason
> this is easier than just "juju bootstrap lxd" from inside the VM?
>
> I suppose our default lxd provider puts the new containers on a NAT
> bridge, though you can reconfigure 'lxdbr0' to bridge your 'eth0' as well.
>
> John
> =:->
>
>
> On Fri, Jun 2, 2017 at 8:33 AM, James Beedy  wrote:
>
>> https://medium.com/@jamesbeedy/using-jaas-to-deploy-lxd-
>> containers-to-virtualbox-vms-on-os-x-a06a8046756a
>>
>> --
>> Juju-dev mailing list
>> juju-...@lists.ubuntu.com
>> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailm
>> an/listinfo/juju-dev
>>
>>
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: JUJU_UNIT_NAME no longer set in env

[Jay Wren]

I was under the impression that `juju run --unit` does run in a hook
context. In fact, the help for `juju help run` explicitly says:

```
Commands run for applications or units are executed in a 'hook context' for
the unit.
```

A quick test:

```
>  juju run --unit u4/0 'echo $JUJU_UNIT_NAME'
u4/0
```

So indeed, $JUJU_UNIT_NAME is set when using `juju run --unit`.

However, another quick test shows that it is not set in the machine case:

```
>  juju run --machine 0 'echo $JUJU_UNIT_NAME'

```

But it is set when using --application, which surprised me:
```
juju run --application u4 'echo $JUJU_UNIT_NAME'
- Stdout: |
u4/0
  UnitId: u4/0
- Stdout: |
u4/1
  UnitId: u4/1
```

James, maybe you have used ssh to get shell on the machine instead of using
juju run?

--
Jay

On Tue, May 23, 2017 at 7:57 AM, Nick Veitch 
wrote:

>
>
> On 23 May 2017 at 11:23, Junien Fridrick 
> wrote:
>
>>
>> You can run some hooks like config-changed with e.g. :
>>
>> $ juju run --unit foo/0 hooks/config-changed
>>
>
> You can run any hook like that, but if it requires a hook context (as in
> the example of trying to read $JUJU_UNIT_NAME ) it won't work. A lot of
> hooks in common charms don't need context, but you can't guarantee that.
>
> N.B. Also when using `juju debug-hooks` once any hook fires and you are in
> the hook environment, you can then execute any hook (not just the one that
> has been triggered), if you need to test them.
>
> --
> Nick Veitch,
> Documentation
> Canonical
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Cannot update bugs-url of promulgated charm

[Jay Wren]

The charm is promulgated and thus its write permissions are set to only
"charmers".

Will a member of charmers please set bugs-url for Merlijn?

Alternatively, you could update a different charm, set its bugs-url and ask
for it to be promulgated.
--
Jay


On Mon, Mar 6, 2017 at 10:55 AM, Merlijn Sebrechts <
merlijn.sebrec...@gmail.com> wrote:

>
> When I want to update the bugs-url of a promulgated charm I get an
> unauthorized error. Is this a bug or am I doing something wrong?
>
> charm set cs:~tengu-team/eclipse-che bugs-url=https://github.com/
> tengu-team/layer-eclipse-che/issues
> ERROR cannot update the set arguments provided: unauthorized: access
> denied for user "merlijn-sebrechts"
>
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Issue replacing the apache2 charm with the haproxy charm in a reverse proxy scenario.

[Jay Wren]

Hello,

I have recently had the need to replace an apache2 ssl termination and
proxy/rewrite reverse proxy system with haproxy. I was hoping I'd be able
to configure the haproxy charm to do all the things that we were doing with
apache2 and its vhost_https_template config. All seemed to work well (huge
thanks to juju and charm authors) until the haproxy ssl termination and
proxy charm was also proxying another haproxy charm. I know this seems
strange, but its an interim config until we collapse our haproxy charms.

Moving from: apache2 -> haproxy-a -> service-a
 -> haproxy-b -> service-b
 -> service-c (direct, no haproxy)
 ... (10 different services)

To:   haproxy -> haproxy-a -> service-a
  -> haproxy-b -> service-b
  -> service-c (direct, no haproxy)
  ... (10 different services)

The issue seems to be that the apache2 charm expects a relation key named
"all_services" and haproxy provides that. haproxy expects a relation key
named "services" but does not provide that.

$ juju run --unit apache2/2 'relation-get -r balancer:93 -
blues-identity-haproxy/2 '
all_services: |
  - server_options:  [check inter 2000 rise 2 fall 5 maxconn 500]
servers:
- - blues-identity-2-8082
  - 10.142.0.16
  - '8082'
  - *id001
service_host: 0.0.0.0
service_name: blues-identity
service_options: [mode http, balance url_param waitid]
service_port: 83
hostname: juju-b4d3a0-41.c.jaas-001.internal
port: "80"
private-address: 10.142.0.15

I've filed a bug, https://bugs.launchpad.net/charm-haproxy/+bug/1664672 and
proposed a fix.
https://code.launchpad.net/~evarlast/charm-haproxy/alias-all_services-services/+merge/317254

Is this intended or is this an oversight or accident?

Thanks,
--
Jay
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Hide resources from showing on the store

[Jay Wren]

+1 its a workaround rather than a feature.

Can we allow publishing charms without resources? This simplifies things
for the charm author for the cases that the charm resource cannot be
uploaded to charmstore.

On Fri, Dec 2, 2016 at 11:59 AM, Gabriel Samfira <
gsamf...@cloudbasesolutions.com> wrote:

> I was not aware that the expected usage was to publish with dummy
> resources. At least, its not reflected in the documentation:
>
> https://jujucharms.com/docs/2.0/developer-resources
>
> Good to know :).
>
> Uploading 0 byte resources sounds like a workaround rather then a feature
> though. Would it not be the same to just allow users to publish a charm
> without attaching any resources at all? That way the charm author can set
> the charm im blocked status if declared resources have not been uploaded.
>
>
> On Vi, 2016-12-02 at 16:41 +, Nate Finch wrote:
>
> I'm with Rick... the expected usage is to publish a dummy resource with
> the charm.  Then, when you deploy, you deploy the real resource from your
> local machine at deploy time.  If you forget, you can always add the
> resource after deploy, as well.  The charm needs to take into consideration
> that someone may forget to include the real resource at deploy time.
>
> Yes, someone could download the dummy resource from the charmstore... I'm
> not sure why this matters, though?  It should be pretty obvious right away
> that it's not the real resource, and the README should also ensure to
> clarify this.
>
> -Nate
>
> On Fri, Dec 2, 2016 at 9:08 AM Gabriel Samfira <
> gsamf...@cloudbasesolutions.com> wrote:
>
> Say you have a charm that deploys a proprietary piece of software. That
> software has an EULA that prohibits anyone from redistributing it. The user
> is essentially forced to go to the software vendor website, login using
> their account, buy the software, and accept the EULA before being able to
> use the software.
>
> In this scenario, we cannot upload that piece of software as a resource
> into the charm store, and set that as the "default" resource for the charm.
> The license for that software does not allow us to. Most users that consume
> such software are aware of this, and is of no surprise to them that they
> have to "own" those resources before they can deploy them.
>
> There are 2 options here:
>
> 1) Canonical is given permission to redistribute the piece of software in
> question (Sharepoint, Exchange, MS SQL ISOs, etc), and validates that the
> user deploying the charm has the right to deploy that software.
> 2) Allow a charm to be published without resources, and validate during
> deployment if the resources are available or not. The charm can handle the
> missing resources and print a nice comprehensive status message. If the
> charm also includes a decent README on how to deploy the service, it should
> be enough to smooth the process. In this scenario, the responsibility of
> making sure all license terms are satisfied, falls onto the user.
>
> It is unfortunate that licensing gets in the way of simplicity, but it is
> an issue that we need to keep in mind.
>
> Cheers,
> Gabriel
>
> On Vi, 2016-12-02 at 13:33 +, Rick Harding wrote:
>
> On Fri, Dec 2, 2016 at 5:49 AM Konstantinos Tsakalozos <
> kos.tsakalo...@canonical.com> wrote:
>
> This works but has a side effect, the dummy binaries are available for
> downloading from the charm's page. Is there a way to hide the resources
> from the charm's readme page?
>
>
> I'm confused why the user cannot know there's a default resource there?
>
> In an ideal world, the author would provide a default resource that at
> least helps guide the user to how to use the charm if they forget or use
> the wrong type of local resource. We want to make sure the user is guided
> on how to get things working as smoothly as possible.
>
> The other side is charm testing. The charm should have enough of what it
> needs in the default resource to perform functional tests to pass via tools
> such as Matrix.
>
> I'm curious how the user even knowing a thing is there is bad for the
> user. At this time we don't have any pattern to assist this.
>
> --
> Juju mailing listj...@lists.ubuntu.com
> Modify settings or unsubscribe at: 
> https://lists.ubuntu.com/mailman/listinfo/juju
>
> --
> Regards,
> Gabriel Adrian Samfira
> Cloudbase Solutions SRL
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/juju
>
> --
> Regards,
> Gabriel Adrian Samfira
> Cloudbase Solutions SRL
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Increasing the size of VERSION in tabular status output

[Jay Wren]

With an accepted minimum value, it seems like $COLUMNS or `stty size` could
be used for variable width status output based on the current terminal
window.

Does the interest in this thread warrant that complexity?

On Mon, Sep 19, 2016 at 10:21 PM, Rick Harding 
wrote:

> 15 seems a bit large. James, it'd be interesting to get a screenshot of a
> full openstack with the messages/etc in place to see how it's sizing up in
> a large complex deploy utilizing the feature fully when rc1 cuts with the
> change.
>
> Thanks
>
> On Mon, Sep 19, 2016 at 7:23 PM Tim Penhey 
> wrote:
>
>> Yesterday we changed the limit to 15 from 7.
>>
>> Tim
>>
>> On 20/09/16 04:41, Rick Harding wrote:
>> > The primary trouble is that we really want to enforce a limit so that
>> > there's room for the arbitrary text at the end of the same line. I think
>> > we could try 10. I do think we need that hard cutoff. If you need to see
>> > the full value going to the json/yaml format output should display the
>> > full value.
>> >
>> > Thanks for the feedback. As it's new it's great to see folks trying it
>> > and bringing up the issues that get hit.
>> >
>> > Rick
>> >
>> > On Mon, Sep 19, 2016 at 12:04 PM John Meinel > > > wrote:
>> >
>> > I'm not sure if the unicode ellipsis would cause table alignment
>> > issues depending on your terminal and font. We could consider it as
>> > it does give quite a few characters back. The longest I can come up
>> > with that doesn't feel just gratuitous is 15
>> > 10.10.10alpha10
>> > But that might be going to far. I do believe the goal was to funnel
>> > people to not giving "postgresql-9.8.0-ia32-icc" sort of version
>> > strings. But it should be useful. 10 characters does seem pretty
>> > good and doesn't cause us to wrap too often.
>> > 1.2.3beta4
>> > Fits in 10, but anything that has 2 digits anywhere would end up
>> > wrapping. It feels a bit odd to force people into 1.2.3b4 though it
>> > does convey the same information it makes you use a string that may
>> > not match the actual upstream nomenclature.
>> >
>> > I guess I could be convinced up to 15 characters but 11 might be an
>> > alternate if we really want people to share the line width but still
>> > allow matching upstream version strings.
>> >
>> > John
>> > =:->
>> >
>> >
>> > On Sep 19, 2016 19:13, "Gregory Lutostanski"
>> > > > > wrote:
>> >
>> > Perhaps also using the utf-8 ellipsis (…) would save some
>> > characters as well.
>> >
>> > --Greg
>> >
>> > On Mon, Sep 19, 2016 at 10:09 AM, James Page
>> > > wrote:
>> >
>> > Hi All
>> >
>> > We've been experimenting with the new
>> > application-version-set feature in Juju 2.0 in the OpenStack
>> > charms team; it provides a much needed way for a charm to
>> > indicate which version of an OpenStack component is deployed
>> > at any given point in time.
>> >
>> > We've come up with an approach that either use the upstream
>> > version of the principle component being deployed, falling
>> > back to the codename for an OpenStack release - for
>> > deployment from source or prior to the packages being
>> > installed for example.
>> >
>> > However, we're finding that 7 chars is a bit limiting in the
>> > default tabular status output - for example:
>> >
>> >   9.0.0~b3 (truncates to 9.0)
>> >   icehouse (truncates to iceh...)
>> >
>> > Could this field be expandable on demand? I think our
>> > longest example would currently be:
>> >
>> >   13.0.0~rc1 (10 chars)
>> >
>> > Cheers
>> >
>> > James
>> >
>> > --
>> > Juju mailing list
>> > Juju@lists.ubuntu.com 
>> > Modify settings or unsubscribe at:
>> > https://lists.ubuntu.com/mailman/listinfo/juju
>> >
>> >
>> >
>> > --
>> > Juju mailing list
>> > Juju@lists.ubuntu.com 
>> > Modify settings or unsubscribe at:
>> > https://lists.ubuntu.com/mailman/listinfo/juju
>> >
>> > --
>> > Juju mailing list
>> > Juju@lists.ubuntu.com 
>> > Modify settings or unsubscribe at:
>> > https://lists.ubuntu.com/mailman/listinfo/juju
>> >
>> >
>> >
>>
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings 

Re: Charm push crashes when uploading big charms

[Jay Wren]

We have made a configuration change which should impact this. If you are
able to try again, please do so.

Also, we are continuing to track this issue and add monitoring and
instrumentation around this area of the charmstore to be better able to
respond to such issues in the future.

Thanks,
--
Jay


On Wed, Jun 22, 2016 at 10:30 AM, Stuart Bishop <stuart.bis...@canonical.com
> wrote:

> On 21 June 2016 at 02:40, Jay Wren <jay.w...@canonical.com> wrote:
> > Thanks for the further testing.
> >
> > Now I'm questioning how in the world I was able to see the same error.
> >
> > I will continue my testing to attempt to reproduce the error.
> >
> > Also, the Apache Timeout 300 should behave exactly as Mark said. I'm
> still
> > trying to find what is causing this failure.
>
> The error from the bug report originates from Squid, which is probably
> buffering the upload. The timeout might happen between Squid and
> Apache if it takes to long to upload and there is no keep-alive
> mechanism in place.
>
> --
> Stuart Bishop <stuart.bis...@canonical.com>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: [Review Queue] rlec, nrpe-external, nagios, squid-reverseproxy, ubuntu-repository-cache, ibm-im

[Jay Wren]

Could you expand on the resource-get inefficiencies?

The resource-get docs say this:

If "resource-get" for a resource has not been run before (for the unit)
>
> then the resource is downloaded from the controller at the revision
>
> associated with the unit's application. That file is stored in the unit's
>
> local cache. If "resource-get" *has* been run before then each
>
> subsequent run syncs the resource with the controller. This ensures
>
> that the revision of the unit-local copy of the resource matches the
>
> revision of the resource associated with the unit's application.
>
>
> Either way, the path provided by "resource-get" references the
>
> up-to-date file for the resource. Note that the resource may get
>
> updated on the controller for the application at any time, meaning the
>
> cached copy *may* be out of date at any time after you call
>
> "resource-get". Consequently, the command should be run at every
>
> point where it is critical that the resource be up to date.
>

Expanding on the inefficiencies given the above optimizations may help
other charm writers be aware of things which may be inefficient.

Thanks,
--
Jay


On Fri, Jun 24, 2016 at 1:59 PM, Kevin Monroe 
wrote:

> Andrew, Cory, Kostas, Pete, and I went through the queue.  Here's what we
> found:
>
>
>-
>
>RLEC (Kostas)
>-
>
>   https://bugs.launchpad.net/charms/+bug/1551133
>   -
>
>   This charm deploys Redis Labs Enterprise Cluster that enables you
>   to install an enterprise-grade cluster for managing and running multiple
>   Redis databases.
>   -
>
>   The charm seems functional. Deploys correctly and scales.
>   -
>
>   There are still a few points that need the attention of the author:
>   -
>
>  Opening ports
>  -
>
>  Better testing
>  -
>
>  Align with the new promulgation process
>  -
>
>nrpe-external (Cory)
>-
>
>
>   
> https://code.launchpad.net/~aluria/charms/precise/nrpe-external-master/donotremove-hostdefs/+merge/290957
>   -
>
>   The PR had already been approved, but to merge in accordance with
>   the new process, it needs to be moved out of the ~charmers namespace.  
> This
>   brought to light that the maintainer is no longer recommending this and
>   users should transition to cs:trusty/nrpe, raising the question of how 
> to
>   handle the transition.  We will apply the PR but move this to 
> ~unmaintained
>   with an announcement to transition and a possible grace period.
>   -
>
>squid-reverseproxy (Pete)
>-
>
>
>   
> https://code.launchpad.net/~dbuliga/charms/trusty/squid-reverseproxy/centos/+merge/287481
>   -
>
>   Marked as “needs fixing” due to failing tests (missing an import on
>   trusty).
>   -
>
>nagios (Pete)
>-
>
>
>   
> https://code.launchpad.net/~dbuliga/charms/trusty/nagios/nagios/+merge/288614
>   -
>
>   Code looks good -- is blocked by nrpe-external being merged,
>   however.
>   -
>
>Ubuntu-repository-cache (Andrew)
>-
>
>
>   
> https://code.launchpad.net/~daniel-thewatkins/charms/trusty/ubuntu-repository-cache/error-message-fix/+merge/292622
>   -
>
>   Due to the large amount of files that need to be synced this fails
>   with a timeout error. Should try to limit the amount of repos synced for
>   the test.
>   -
>
>IBM Installation Manager (Kevin)
>-
>
>   https://bugs.launchpad.net/charms/+bug/1575746
>   -
>
>   We noticed our previous MP was inefficiently calling resource-get
>   multiple times for the fixpack (which is a real problem with large
>   resources). Fixed.
>   -
>
>   IBM found a bug in our previous MP where we sentry’d the wrong
>   status message in the ibm-im amulet test.  Fixed.
>   - Awaiting merge decision.
>
>
> Let us know if you have any questions. Thanks!
> -Kevin
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm push crashes when uploading big charms

[Jay Wren]

Thanks for the further testing.

Now I'm questioning how in the world I was able to see the same error.

I will continue my testing to attempt to reproduce the error.

Also, the Apache Timeout 300 should behave exactly as Mark said. I'm still
trying to find what is causing this failure.


On Mon, Jun 20, 2016 at 9:49 PM, Merlijn Sebrechts <
merlijn.sebrec...@gmail.com> wrote:

> So if that's 300 days, that should give you plenty of time ;)
>
> 2016-06-20 20:38 GMT+02:00 Merlijn Sebrechts <merlijn.sebrec...@gmail.com>
> :
>
>> From the bug report: "The timeout is apache setting `Timeout` which
>> defaults to 300."
>>
>> https://bugs.launchpad.net/ubuntu/+source/charm/+bug/1592822
>>
>> 2016-06-20 20:32 GMT+02:00 Tom Barber <t...@analytical-labs.com>:
>>
>>> If I had to upload 270mb from my home I'd be waiting 3 weeks. what's
>>> the timeout set to? ;)
>>> On 20 Jun 2016 19:30, "Merlijn Sebrechts" <merlijn.sebrec...@gmail.com>
>>> wrote:
>>>
>>>> Thanks for the explanation, Jay.
>>>>
>>>> I did some further testing. Charm upload fails for a 270MB Charm both
>>>> from my home, my work and our datacenter.
>>>>
>>>>- The datacenter is connected directly to Belnet (upload bandwith
>>>>~300Mbit/s).
>>>>- My upload bandwidth at home is ~ 3 Mbps (speedtest.net) although
>>>>during upload, system monitor shows ~400KiB/s.
>>>>
>>>> This causes me to think there is more at play here then large file +
>>>> slow internet... Let me know if I can help to further debug this problem.
>>>>
>>>>
>>>> As an aside; I don't consider 270MB to be that large. Some examples:
>>>>
>>>>
>>>>- Kubernetes is ~1G
>>>>- Ubuntu docker base image is ~200MB
>>>>
>>>>
>>>> I think this is stuff we should be able to handle...
>>>>
>>>> 2016-06-20 18:41 GMT+02:00 Jay Wren <jay.w...@canonical.com>:
>>>>
>>>>> Yes, files are broken up into many TCP packets, and they are all
>>>>> transmitted over a single TCP connection. TCP is a complex protocol which
>>>>> is well documented, so I'll not repeat that here. If you want lots of
>>>>> details, wikipedia is not bad:
>>>>> https://en.wikipedia.org/wiki/Transmission_Control_Protocol#Protocol_operation
>>>>>
>>>>> In the abstract, when you connect to a server using TCP it is
>>>>> identified by a 4-tuple of source address, source port, target ip address,
>>>>> target port. These connections consume server resources and indeed a
>>>>> connection exhaustion is a popular denial of service attack.
>>>>>
>>>>> You are getting a tcp timeout due to of file size because the time it
>>>>> takes to send the entire content is longer than the TCP connection 
>>>>> timeout.
>>>>>
>>>>> Yes, the resource upload command to charmstore will also be affected
>>>>> by this. Luckily, resources also have the ability to be uploaded
>>>>> specifically to a model, which might have greater network data rates from
>>>>> the resource uploader.
>>>>>
>>>>> --
>>>>> Jay
>>>>>
>>>>>
>>>>> On Mon, Jun 20, 2016 at 7:04 PM, Merlijn Sebrechts <
>>>>> merlijn.sebrec...@gmail.com> wrote:
>>>>>
>>>>>> Thanks for looking into this! I'll try the compression and see if
>>>>>> that works.
>>>>>>
>>>>>> Just curious; why does filesize affect tcp connection timeout? Aren't
>>>>>> the files broken up into a bunch of smaller tcp packets? Filesize should
>>>>>> only affect the number of tcp packets, not the size of the tcp packets? 
>>>>>> So
>>>>>> getting a tcp timeout because of filesize seems strange to me... Any idea
>>>>>> what exactly goes wrong here?
>>>>>>
>>>>>> Also, now that I think of it, the resource upload command might also
>>>>>> be affected by this, if it uses the same library and similar backend
>>>>>> infrastructure? I'll test this out.
>>>>>>
>>>>>>
>>>>>> Op maandag 20 juni 2016 heeft Jay Wren <jay.w...@canonical.com> het
>>>>>> volgende geschr

Re: Charm push crashes when uploading big charms

[Jay Wren]

Yes, files are broken up into many TCP packets, and they are all
transmitted over a single TCP connection. TCP is a complex protocol which
is well documented, so I'll not repeat that here. If you want lots of
details, wikipedia is not bad:
https://en.wikipedia.org/wiki/Transmission_Control_Protocol#Protocol_operation

In the abstract, when you connect to a server using TCP it is identified by
a 4-tuple of source address, source port, target ip address, target port.
These connections consume server resources and indeed a connection
exhaustion is a popular denial of service attack.

You are getting a tcp timeout due to of file size because the time it takes
to send the entire content is longer than the TCP connection timeout.

Yes, the resource upload command to charmstore will also be affected by
this. Luckily, resources also have the ability to be uploaded specifically
to a model, which might have greater network data rates from the resource
uploader.

--
Jay


On Mon, Jun 20, 2016 at 7:04 PM, Merlijn Sebrechts <
merlijn.sebrec...@gmail.com> wrote:

> Thanks for looking into this! I'll try the compression and see if that
> works.
>
> Just curious; why does filesize affect tcp connection timeout? Aren't the
> files broken up into a bunch of smaller tcp packets? Filesize should only
> affect the number of tcp packets, not the size of the tcp packets? So
> getting a tcp timeout because of filesize seems strange to me... Any idea
> what exactly goes wrong here?
>
> Also, now that I think of it, the resource upload command might also be
> affected by this, if it uses the same library and similar backend
> infrastructure? I'll test this out.
>
>
> Op maandag 20 juni 2016 heeft Jay Wren <jay.w...@canonical.com> het
> volgende geschreven:
> > Hello Merlijn,
> > I can replicate the problem and I can work around it by using a faster
> internet connection.
> > At some point, tcp connections have to time out. I can only replicate
> the issue when that timeout is reached. If you have the means to relocate
> to a faster internet connection temporarily for pushing to charmstore,
> please do. You might also try recompressing any items in the charm using a
> higher compression level. xz -9 instead of gzip -3 or whatever things may
> be using now.
> > We are aware this is a poor longterm solution. We are investigating
> better solutions for uploads. As you've mentioned, resources will also help
> the situation.
> > I am sorry that I do not have a better solution.
> > --
> > Jay
> > On Fri, Jun 17, 2016 at 4:29 PM, Rick Harding <
> rick.hard...@canonical.com> wrote:
> >>
> >> Merlijn, thanks. I'm going to bet there's an issue with http request
> sizes for the charmstore that the charm command talks do as we've got some
> layers (Apache, Squid) in front of the actual application. The team is
> looking into it. Thanks for giving us the heads up.
> >> On Wed, Jun 15, 2016 at 10:28 AM Merlijn Sebrechts <
> merlijn.sebrec...@gmail.com> wrote:
> >>>
> >>> Hi all
> >>>
> >>> I've hit a roadblock in setting up my CI pipeline. I have a charm with
> a Java sdk blob of ~200MB. Pushing this charm to the store causes the tool
> to crash. I put a bug report here:
> https://bugs.launchpad.net/juju/+bug/1592822
> >>> Juju resources would fix my problem, but then I'd need to move to Juju
> 2.0 and I'm not ready to do that yet.
> >>>
> >>>
> >>> Kind regards
> >>> Merlijn Sebrechts
> >>> --
> >>> Juju mailing list
> >>> Juju@lists.ubuntu.com
> >>> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
> >>
> >> --
> >> Juju mailing list
> >> Juju@lists.ubuntu.com
> >> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
> >>
> >
> >
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm push crashes when uploading big charms

[Jay Wren]

Hello Merlijn,

I can replicate the problem and I can work around it by using a faster
internet connection.

At some point, tcp connections have to time out. I can only replicate the
issue when that timeout is reached. If you have the means to relocate to a
faster internet connection temporarily for pushing to charmstore, please
do. You might also try recompressing any items in the charm using a higher
compression level. xz -9 instead of gzip -3 or whatever things may be using
now.

We are aware this is a poor longterm solution. We are investigating better
solutions for uploads. As you've mentioned, resources will also help the
situation.

I am sorry that I do not have a better solution.
--
Jay

On Fri, Jun 17, 2016 at 4:29 PM, Rick Harding 
wrote:

> Merlijn, thanks. I'm going to bet there's an issue with http request sizes
> for the charmstore that the charm command talks do as we've got some layers
> (Apache, Squid) in front of the actual application. The team is looking
> into it. Thanks for giving us the heads up.
>
> On Wed, Jun 15, 2016 at 10:28 AM Merlijn Sebrechts <
> merlijn.sebrec...@gmail.com> wrote:
>
>> Hi all
>>
>>
>> I've hit a roadblock in setting up my CI pipeline. I have a charm with a
>> Java sdk blob of ~200MB. Pushing this charm to the store causes the tool to
>> crash. I put a bug report here:
>> https://bugs.launchpad.net/juju/+bug/1592822
>>
>> Juju resources would fix my problem, but then I'd need to move to Juju
>> 2.0 and I'm not ready to do that yet.
>>
>>
>>
>> Kind regards
>> Merlijn Sebrechts
>> --
>> Juju mailing list
>> Juju@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju
>>
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Question for cosmetic and understandability of breadcrumbs in github

[Jay Wren]

If it helps, the charm set command supports arbitrary key values to be
stored in extra-info in the charm store.

e.g.

$ charm set cs:~evarlast/trusty/parse-server-0 layer-x=rev1 layer-y=rev2

Then this will be displayed in extrainfo:

$ charm show cs:~containers/trusty/swarm extra-info
extra-info:
  layer-x: rev1
  layer-y: rev2

After pushing, you could use set to save the revisions of what was used to
build the charm.

--
Jay

On Thu, Jun 16, 2016 at 8:51 PM, Charles Butler <
charles.but...@canonical.com> wrote:

> Greetings,
>
> I deposit many of my layers in GitHub, and one of the things  I've been
> striving to do is keep tag releases at the revisions i cut a charm release
> for a given channel. As we know, the default channel is seen by no-one, and
> runs in increments of n+1.
>
> My prior projects i've been following semver for releases, but that has
> *nothing* in terms of a breadcrumb trail back to the store.
>
> Would it be seen as good practice to tag releases - on the top most layer
> of a charm - with what charm release its coordinated with?
>
> Given the scenario that i'm ready to release swarm, and lets assume that
> to date i haven't tagged any releases in the layer repository:
>
> charm show cs:~containers/trusty/swarm revision-info
> revision-info:
>   Revisions:
>   - cs:~containers/trusty/swarm-2
>   - cs:~containers/trusty/swarm-1
>   - cs:~containers/trusty/swarm-0
>
> I see that i'm ready to push swarm-3 to the store:
>
> git tag 3
> git push origin --tags
>
> I can now correlate the source revision to what i've put in my account on
> the store, but this does not account for promulgation (which has an
> orthogonal revision history), and mis-match of those id's.
>
> I think this can simply be documented that tags track
> <>/<> pushes, and to correlate source with release, to use
> the method shown above to fetch release info.
>
> Does this sound useful/helpful or am I being pedantic? (I say this because
> Kubernetes touches ~ 7 layers, and it gets confusing keeping everything up
> to date locally while testing, and then again re-testing with
> --no-local-layers to ensure our repositories are caught up with current
> development work. (Cant count the number of open pull requests hanging
> waiting for review because we've moved to the next hot-ticket item)
>
> All the best,
>
> Charles
> --
> Juju Charmer
> Canonical Group Ltd.
> Ubuntu - Linux for human beings | www.ubuntu.com
> Juju - The fastest way to model your service | www.jujucharms.com
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: error with mariadb

[Jay Wren]

Clicking that URL and reading the InRelease file, you can see that the
mirror at http://ftp.osuosl.org/pub/mariadb/repo/5.5/ubuntu/ does not have
ppc64el packages.

A quick search at packages.ubuntu.com shows that trusty has mariadb 5.5
packaged.
http://packages.ubuntu.com/search?keywords=mariadb=names=trusty=all

I took a quick look at the mariadb charm at https://jujucharms.com/mariadb/
and I do not see a PPA or other repository config value. Are you using that
charm or some other charm?

In Daniel's reply he wrote "I can confirm that the MariaDB Foundation
repositories do have
packages for MariaDB 5.5 for Trusty on Power 8 Little Endian."

It does not sound like  http://ftp.osuosl.org/pub/mariadb/repo/5.5/ubuntu/
is a full mirror of the MariaDB Foundation repositories.

A list of mirrors is here:
https://downloads.mariadb.org/mariadb/repositories/#mirror=jaleco=Ubuntu_release=trusty--ubuntu_trusty=5.5

I selected each mirror and each one appears to only mirror [arch=amd64,i386]
.

I'm afraid I don't know the next step to help you, but I hope this
information is helpful. Maybe Daniel could reply with the location of the
Maria DB Foundation repositories which include ppc64le trusty packages.

On Mon, Jun 6, 2016 at 8:55 AM, Rajith P Venkata 
wrote:

> Hi
>
> Thanks for below information, I tried with below options, after mariadb
> 5.5  got installed, I tried install my charm but I am getting error: INFO
> install W: Failed to fetch
> http://ftp.osuosl.org/pub/mariadb/repo/5.5/ubuntu/dists/trusty/InRelease 
> Unable
> to find expected entry 'main/binary-ppc64el/Packages' in Release file
> (Wrong sources.list entry or malformed file)
>
> Please check below logs .
>
>
>
>
> Rajith
>
> IBM AIX Certified, OCPCertified
> 
>
> Cell- 9901966577
> Email: rajith...@in.ibm.com
>
>
>
> From:Daniel Bartholomew 
> To:Rajith P Venkata/India/IBM@IBMIN
> Cc:juju 
> Date:03-06-16 10:55 PM
> Subject:Re: error with mariadb
> --
>
>
>
> On Fri, Jun 3, 2016 at 6:18 AM, Rajith P Venkata 
> wrote:
> > I am trying to use  Mariadb 5.5 on  Ubuntu 14.x on Power 8 Little Endian
> > with this we are getting error as unexpected token,
> > Token I have used form https://mariadb.com/my_portal
> >
> > With verision Mariadb 10.05 we do not get this error but the plugin we
> are
> > installing needs Mariadb 5.5.
> >
> > Please refer the logs below and let us know if Mariadb 5.5 works on
> Ubuntu
> > 14.x on Power 8 Little Endian
>
> MariaDB 5.5 does work on Ubuntu 14.x on Power 8 Little Endian.
> However, after looking at the logs, I'm not sure what your error is.
>
> In the unit-mariadb-5.log file, I see one error where something is
> trying to connect to MariaDB when MariaDB isn't running, but I'm not
> sure exactly what is trying to connect. A few lines further down in
> the log MariaDB is started again, so if whatever was trying to access
> MariaDB had waited a few seconds I would assume the connection attempt
> would have succeeded. During installation and upgrades MariaDB is
> stopped and started at various points as needed to apply updates to
> internal system tables and other tasks.
>
> In the unit-ibm-platform-rtm-4.log I see that line 203 of the
> ibm-prtm.sh script has a syntax error, but I don't know anything about
> that script that would enable me to comment further. I also see that
> the script runs a ibm-prtm-mariadb.available test several times, but
> again I don't know anything about that test or what it is checking
> for.
>
> I don't know if it will help you or not, but I recently pushed some
> updates to my personal launchpad tree -
> lp:~dbart/charms/trusty/mariadb/trunk - that sets the MariaDB charm to
> use the MariaDB Foundation repositories by default if the MariaDB Inc
> EULA and token are not provided as specified in the README. This new
> version of the charm is not in the charm store yet, but hopefully it
> will be soon.
>
> I can confirm that the MariaDB Foundation repositories do have
> packages for MariaDB 5.5 for Trusty on Power 8 Little Endian. The only
> caveat I would add is that you'll get much better performance on Power
> 8 using MariaDB 10.0 or 10.1.
>
>
> Also, the choice of which series to use should be changed using a
> local config.yaml file with the series variable set to one of "5.5",
> "10.0", or "10.1" (the default in the charm store is currently 10.0,
> but my new update bumps the default up to 10.1). For example:
>
> mariadb:
>series: "5.5"
>
> When using my repository locally I can then deploy MariaDB 5.5 with:
> juju deploy --config ./mariadb.yaml
> --repository="/home/${USER}/src/charms/" local:trusty/mariadb
>
>
> And FYI, using something like:
>
> juju set mariadb series="5.5"
>
> ...after the initial deployment to downgrade MariaDB from the default
> series to 5.5 does not work. It has to be 

Re: Anyone working on HTTP/2 in apache2 charm?

[Jay Wren]

tl;dr: yes, simply `juju set apache2 apt-source:ppa:ondrej/apache2` and `juju
set apache2 enable_modules=ssl,http2`

Given that the charm installs apache2 from ubuntu repos, its pretty easy to
extend the charm to allow it to enabled a PPA. I did this back in October:
http://jrwren.wrenfam.com/blog/2015/10/26/juju-makes-getting-to-http2-easier/

Sometime later that was updated to be more friendly and merged into the
apache2 trusty charm: https://jujucharms.com/apache2/trusty/20

AFAIK, Apache2 shipped in Xenial without http2 enabled for security
reasons, it is not the default for good reasons. For non-production use,
the ondrej/apache2 PPA works very well.

--
Jay Wren


On Mon, May 9, 2016 at 12:22 PM, Robin Winslow <ro...@canonical.com> wrote:

> I spent the weekend playing around with getting mod_http2
> <https://httpd.apache.org/docs/2.4/mod/mod_http2.html> working in Apache2
> on my laptop.
>
> Although HTTP/2 <https://http2.github.io/> is supported in the xenial
> version of nginx, Apache2 in Xenial doesn't currently
> <https://bugs.launchpad.net/ubuntu-release-notes/+bug/1531864>
> support mod_http2. To get it, you have to compile Apache yourself
> <https://icing.github.io/mod_h2/howto.html> with the --enable-http2
> option.
>
> Does anyone know of any existing work on adding HTTP/2 support to the
> Apache2 charm?
>
> If not, I may have a go at extending it myself.
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: jujucharms.com not able to ingest charms and bundles at this time.

[Jay Wren]

I'm happy to report the team has worked with webops and resolved
delayed ingestion..

Charm ingestion is running and the latest charms should all be available.
If you find any issues please let me know.

Thank you everyone for your patience while we got this fixed up and working
smoothly again.

On Mon, Jan 11, 2016 at 12:24 PM, Jay Wren <jay.w...@canonical.com> wrote:
> This morning the Juju UI Engineering team noticed that ingestion from
> launchpad.net to jujucharms.com has not been working since December
> 17th.
>
> The issue is critical and the team is investigating immediately. At
> this point we do not have a firm date and time for when full service
> will be restored.
>
> If you have any questions please don't hesitate to ping me on irc (jrwren)
> or via email and I will do my best to help in any way I can.
>
> Thank you all for your patience and understanding.
> --
> Jay Wren
> Juju UI Engineering

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

charm-helpers fetch apt_install optimization

[Jay Wren]

Greetings,

I wanted to bring attention to a pending MR
https://code.launchpad.net/~evarlast/charm-helpers/use-no-install-recommends/+merge/242270
which I feel can help charm-helpers.

By not installing recommended packages, charms will install packages
faster, with less unneeded packages.

This leads to faster deploys and faster experience for new users and an
overall better feeling about juju.

On a similar subject, I am asking for feedback from charm-helpers users
regarding apt related activity. I feel it might make sense use apt_pkg for
as many operations as possible. Would a merge request of this nature be
considered or is fork and exec to the more familiar system commands
preferred?

Thanks,
--
Jay
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju