Availability of alpha Easy Juju on Azure disk image
Dear All, Yesterday we released an alpha of a new Ubuntu image on MS Azure VM Depot: https://vmdepot.msopentech.com/Vhd/Show?vhdId=50248 If you use that image and spin a VM with it, you'll be able to upload your Azure .publishsettings file to its web interface. From that moment, the VM will boostrap a Juju environment, install the Juju GUI. The main web page auto refreshes, and presents the link to the Juju GUI with a password when it's ready. The whole process from upload to Juju GUI takes about 10min to complete, and does not require any knowledge of Juju or Ubuntu to start playing. It leaves you with a fully functional Juju environment using your default subscription as your main Azure provider. We hope you enjoy that new and easy way to start with Juju. We aim at making this image available from the Marketplace when we gather enough feedback and fix bugs that remain. The whole code and explanations are available on https://github.com/SaMnCo/juju-azure. Please use that repo to send feedback while we make it better. You can also answer to this thread on the list to ask any question or feature request you may have. Best, Samuel -- Samuel Cozannet Cloud, Big Data and IoT Strategy Team Business Development - Cloud and ISV Ecosystem Changing the Future of Cloud Ubuntu http://ubuntu.com / Canonical UK LTD http://canonical.com / Juju https://jujucharms.com samuel.cozan...@canonical.com mob: +33 616 702 389 skype: samnco Twitter: @SaMnCo_23 -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Availability of alpha Easy Juju on Azure disk image
Hey Andrew, I agree with you on the feedback, since it's an HTML page at the moment it's hard to incorporate those items, but I've forked the repo and started building a lightweight Python app to provide that feedback to the user. Marco On Tue Feb 10 2015 at 9:37:48 PM Andrew Wilkins andrew.wilk...@canonical.com wrote: Hi Samuel, Looks neat. A few things: 1. Once the VM is ready, the entire landing page keeps refreshing at 1s intervals. That doesn't leave a lot of time to add the .publishsettings file and upload. 2. There's no feedback to say whether or not the .publishsetttings file has been uploaded, and whether bootstrapping is underway. I guess I did it twice because of point 1, because now I have two juju bootstrap processes on the machine :) 3. When the GUI eventually came up, it wouldn't accept the password that the page displayed. This is related to point 2: the page gave me the link to the GUI for one env, and the password for the other. It'd be great if the upload credentials bit greyed out once uploaded, and then under 3. Wait for a few minutes... there was something describing the current status. e.g. Installing Juju, Deploying Juju GUI. Cheers, Andrew On Tue, Feb 10, 2015 at 11:01 PM, Samuel Cozannet samuel.cozan...@canonical.com wrote: Dear All, Yesterday we released an alpha of a new Ubuntu image on MS Azure VM Depot: https://vmdepot.msopentech.com/Vhd/Show?vhdId=50248 If you use that image and spin a VM with it, you'll be able to upload your Azure .publishsettings file to its web interface. From that moment, the VM will boostrap a Juju environment, install the Juju GUI. The main web page auto refreshes, and presents the link to the Juju GUI with a password when it's ready. The whole process from upload to Juju GUI takes about 10min to complete, and does not require any knowledge of Juju or Ubuntu to start playing. It leaves you with a fully functional Juju environment using your default subscription as your main Azure provider. We hope you enjoy that new and easy way to start with Juju. We aim at making this image available from the Marketplace when we gather enough feedback and fix bugs that remain. The whole code and explanations are available on https://github.com/SaMnCo/juju-azure. Please use that repo to send feedback while we make it better. You can also answer to this thread on the list to ask any question or feature request you may have. Best, Samuel -- Samuel Cozannet Cloud, Big Data and IoT Strategy Team Business Development - Cloud and ISV Ecosystem Changing the Future of Cloud Ubuntu http://ubuntu.com / Canonical UK LTD http://canonical.com / Juju https://jujucharms.com samuel.cozan...@canonical.com mob: +33 616 702 389 skype: samnco Twitter: @SaMnCo_23 -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/ mailman/listinfo/juju -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Availability of alpha Easy Juju on Azure disk image
Hi Samuel, Looks neat. A few things: 1. Once the VM is ready, the entire landing page keeps refreshing at 1s intervals. That doesn't leave a lot of time to add the .publishsettings file and upload. 2. There's no feedback to say whether or not the .publishsetttings file has been uploaded, and whether bootstrapping is underway. I guess I did it twice because of point 1, because now I have two juju bootstrap processes on the machine :) 3. When the GUI eventually came up, it wouldn't accept the password that the page displayed. This is related to point 2: the page gave me the link to the GUI for one env, and the password for the other. It'd be great if the upload credentials bit greyed out once uploaded, and then under 3. Wait for a few minutes... there was something describing the current status. e.g. Installing Juju, Deploying Juju GUI. Cheers, Andrew On Tue, Feb 10, 2015 at 11:01 PM, Samuel Cozannet samuel.cozan...@canonical.com wrote: Dear All, Yesterday we released an alpha of a new Ubuntu image on MS Azure VM Depot: https://vmdepot.msopentech.com/Vhd/Show?vhdId=50248 If you use that image and spin a VM with it, you'll be able to upload your Azure .publishsettings file to its web interface. From that moment, the VM will boostrap a Juju environment, install the Juju GUI. The main web page auto refreshes, and presents the link to the Juju GUI with a password when it's ready. The whole process from upload to Juju GUI takes about 10min to complete, and does not require any knowledge of Juju or Ubuntu to start playing. It leaves you with a fully functional Juju environment using your default subscription as your main Azure provider. We hope you enjoy that new and easy way to start with Juju. We aim at making this image available from the Marketplace when we gather enough feedback and fix bugs that remain. The whole code and explanations are available on https://github.com/SaMnCo/juju-azure. Please use that repo to send feedback while we make it better. You can also answer to this thread on the list to ask any question or feature request you may have. Best, Samuel -- Samuel Cozannet Cloud, Big Data and IoT Strategy Team Business Development - Cloud and ISV Ecosystem Changing the Future of Cloud Ubuntu http://ubuntu.com / Canonical UK LTD http://canonical.com / Juju https://jujucharms.com samuel.cozan...@canonical.com mob: +33 616 702 389 skype: samnco Twitter: @SaMnCo_23 -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Unable to deploy a working 10 node hadoop cluster
Hi, I'm trying to deploy a basic hadoop cluster on Amazon (AWS). It should have just 10 nodes to simply run hadoop map-reduce jobs and just store data on hdfs. Nothing else. No hive or spark or anything. These are the commands I enter. juju quickstart juju deploy hdp-hadoop yarn-hdfs-master juju deploy hdp-hadoop compute-node juju add-unit -n 10 compute-node juju add-relation yarn-hdfs-master:namenode compute-node:datanode juju add-relation yarn-hdfs-master:resourcemanager compute-node:nodemanager In 'juju status' I can see all the nodes are being added and I wait until all their statuses are 'running'. If I 'juju ssh' to *any* of the compute-node machines I cannot list any hdfs directories, and get this message root@ip-172-31-28-205:~# su hdfs hdfs@ip-172-31-28-205:/home/ubuntu$ hdfs dfs -ls / ls: Incomplete HDFS URI, no host: hdfs://TODO-NAMENODE-HOSTNAME:PORT hdfs@ip-172-31-28-205:/home/ubuntu$ Also, there is no 'DataNode' process running on the machine, which would be needed to access HDFS. Am I doing something wrong or am I meant to edit the 'hdfs-site.xml' file myself ? On all 10 machines ? Also, if I 'juju ssh' onto the yarn-hdfs-master/0 machine and try to run a hdfsadmin -report, it tells me that hdfs has no data-nodes running (see below) - so when I try to put data on to hdfs it fails with an error message 'There are 0 datanode(s) running'. hdfs@ip-172-31-21-161:/home/ubuntu$ hdfs dfsadmin -report Configured Capacity: 0 (0 B) Present Capacity: 0 (0 B) DFS Remaining: 0 (0 B) DFS Used: 0 (0 B) DFS Used%: NaN% Under replicated blocks: 0 Blocks with corrupt replicas: 0 Missing blocks: 0 - Datanodes available: 0 (0 total, 0 dead) I don't understand if I am doing something wrong. What is the recommended way for deploying a hadoop and hdfs cluster using juju ? Thankyou for any help, Ken -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
On not copy-pasting state and params struct conversions
Hi all, Anastasia raised an issue in http://reviews.vapour.ws/r/885/ about how to cut down on struct conversions between params, state, and domain packages. In this case we're talking about storage. The following API server facades currently participate in storage: - client - storage - provisioner - diskmanager (to be renamed, this lists block devices on machines) - diskformatter - storageworker (to be renamed, this is the dynamic storage provisioner) Each facade have some overlap in dealing with storage entities, e.g. the diskformatter and diskmanager each need to deal with block devices. This leads to much duplication of struct copying/conversion code when toing and froing between state and clients. I don't want to go adding conversion code to the params, state or storage packages, as they really shouldn't have dependencies on each other. Does anyone have a good idea about where to put this common functionality? Maybe api/common/storage, apiserver/common/storage? Does not appeal, but I can't think of a better option. Cheers, Andrew -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: On not copy-pasting state and params struct conversions
So, I think you *do* need conversion code specific to state -- the methods exported from state should be in terms of the storage types, because they're the language we've chosen to model storage, so there needs to be conversion code in state (or possibly in an internal package -- but either way there's no reason to expose it to anyone else). WRT params, though, we already have two groups of client packages, so the same forces don't quite apply: and I still prefer that we *not* import other packages into params [0]. How about a single params/convert package? Cheers William [0] this is because the temptation to use a storage.Foo in place of a params.Foo has historically been overwhelming (and while it's *very dangerous* to do that it's hard to make the danger sufficiently obvious to dissuade distracted devs from doing so). On Tue, Feb 10, 2015 at 11:07 AM, Andrew Wilkins andrew.wilk...@canonical.com wrote: Hi all, Anastasia raised an issue in http://reviews.vapour.ws/r/885/ about how to cut down on struct conversions between params, state, and domain packages. In this case we're talking about storage. The following API server facades currently participate in storage: - client - storage - provisioner - diskmanager (to be renamed, this lists block devices on machines) - diskformatter - storageworker (to be renamed, this is the dynamic storage provisioner) Each facade have some overlap in dealing with storage entities, e.g. the diskformatter and diskmanager each need to deal with block devices. This leads to much duplication of struct copying/conversion code when toing and froing between state and clients. I don't want to go adding conversion code to the params, state or storage packages, as they really shouldn't have dependencies on each other. Does anyone have a good idea about where to put this common functionality? Maybe api/common/storage, apiserver/common/storage? Does not appeal, but I can't think of a better option. Cheers, Andrew -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: A home for underappreciated errors
On Tue, Feb 10, 2015 at 12:13 PM, William Reade william.re...@canonical.com wrote: really ought to include a bunch more mechanism for finding out *precisely* what should have been assigned to what To be clear: I do not think we should do this, because the interpretation of those entities would themselves be domain-specific -- if client code is written to expect a unit id and ends up suddenly seeing a volume id, the *best* case is that it fails -- the worst is that it keeps on running with bad data and smearing the effects of the bug further through the system with arbitrary further fallout. +100 to domain-specific errors. -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: A home for underappreciated errors
On Tue, Feb 10, 2015 at 6:16 PM, roger peppe roger.pe...@canonical.com wrote: On 10 February 2015 at 10:06, Andrew Wilkins andrew.wilk...@canonical.com wrote: On Tue, Feb 10, 2015 at 5:57 PM, roger peppe roger.pe...@canonical.com wrote: On 10 February 2015 at 08:55, Andrew Wilkins andrew.wilk...@canonical.com wrote: Hi all, Ian raised a good point in http://reviews.vapour.ws/r/885/ (caution advised, may cause eyebleed) about a change I made to errors: https://github.com/juju/errors/pull/17 The NotAssigned error, which previously was only raised by state.Unit.AssignedMachineId, is now also raised by state.Volume.StorageInstance. I removed the error from the state package so we didn't have apiserver/common poking its head in there, and moved it over to the juju/errors package. The issue Ian raised is that juju/errors should contain relatively generic error types, and we should keep domain-specific things elsewhere. Examples are NotAssigned, and NotProvisioned. We're thinking of moving these domain-specific error types into a new package, juju/juju/errors. What do you think about this? I agree with Ian here. Personally, I think that specific error types/values work well when defined in or close to the module that returns them, which is aligned to my general feeling that the set of possible returned errors is part of the contract of the method it was returned from. For example, the mgo package returns mgo.ErrNotFound when an item isn't found, and that's fine - we translate from that error to a more local not-found error when necessary. For errors that pass over an API, they can be considered a part of the API and defined along with the other API types. So, rather than a single package for juju-related errors, I would suggest that domain-specific errors are defined in packages close to the domain in question rather than in a single package used by everything. To my mind this is a more modular and scalable approach. That all sounds good, but it doesn't work very well with the existing error code thingy in apiserver/common, unless we're okay with apiserver/common poking into state. I'm not a fan of that. Perhaps we just need a state-internal package which registers predicates in apiserver/common to do that magic. That wouldn't be so bad IMO. I don't understand. Can't you just define NotAssignedError in state and have the usual predicate operations in apiserver/common ? (you can't import apiserver/common from state BTW - you'd get a cycle) Yeah, I didn't think that through well. Long day. Having it state dependency from apiserver/common is, as William points out, fine. It's all going to state anyway. -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: On not copy-pasting state and params struct conversions
I think that the danger here is one of backwards incompatibility, and that's a danger that's possible to alleviate with some tooling support. Writing tons of boilerplate code to manually convert between types at different levels has always seemed like a poor use of developer resources to me (and opens up more possibility for conversion errors). When things *do* change in a backwardly incompatible way, then of course conversion code would need to be written, but if there is a tool that can automatically test that things remain compatible, this could be only a small percentage of what is being done up front now. cheers, rog. On 10 February 2015 at 09:07, Andrew Wilkins andrew.wilk...@canonical.com wrote: Hi all, Anastasia raised an issue in http://reviews.vapour.ws/r/885/ about how to cut down on struct conversions between params, state, and domain packages. In this case we're talking about storage. The following API server facades currently participate in storage: - client - storage - provisioner - diskmanager (to be renamed, this lists block devices on machines) - diskformatter - storageworker (to be renamed, this is the dynamic storage provisioner) Each facade have some overlap in dealing with storage entities, e.g. the diskformatter and diskmanager each need to deal with block devices. This leads to much duplication of struct copying/conversion code when toing and froing between state and clients. I don't want to go adding conversion code to the params, state or storage packages, as they really shouldn't have dependencies on each other. Does anyone have a good idea about where to put this common functionality? Maybe api/common/storage, apiserver/common/storage? Does not appeal, but I can't think of a better option. Cheers, Andrew -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: A home for underappreciated errors
On Tue, Feb 10, 2015 at 5:57 PM, roger peppe roger.pe...@canonical.com wrote: On 10 February 2015 at 08:55, Andrew Wilkins andrew.wilk...@canonical.com wrote: Hi all, Ian raised a good point in http://reviews.vapour.ws/r/885/ (caution advised, may cause eyebleed) about a change I made to errors: https://github.com/juju/errors/pull/17 The NotAssigned error, which previously was only raised by state.Unit.AssignedMachineId, is now also raised by state.Volume.StorageInstance. I removed the error from the state package so we didn't have apiserver/common poking its head in there, and moved it over to the juju/errors package. The issue Ian raised is that juju/errors should contain relatively generic error types, and we should keep domain-specific things elsewhere. Examples are NotAssigned, and NotProvisioned. We're thinking of moving these domain-specific error types into a new package, juju/juju/errors. What do you think about this? I agree with Ian here. Personally, I think that specific error types/values work well when defined in or close to the module that returns them, which is aligned to my general feeling that the set of possible returned errors is part of the contract of the method it was returned from. For example, the mgo package returns mgo.ErrNotFound when an item isn't found, and that's fine - we translate from that error to a more local not-found error when necessary. For errors that pass over an API, they can be considered a part of the API and defined along with the other API types. So, rather than a single package for juju-related errors, I would suggest that domain-specific errors are defined in packages close to the domain in question rather than in a single package used by everything. To my mind this is a more modular and scalable approach. That all sounds good, but it doesn't work very well with the existing error code thingy in apiserver/common, unless we're okay with apiserver/common poking into state. I'm not a fan of that. Perhaps we just need a state-internal package which registers predicates in apiserver/common to do that magic. That wouldn't be so bad IMO. It's also the standard Go idiom. cheers, rog. -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: On not copy-pasting state and params struct conversions
On Tue, Feb 10, 2015 at 5:44 PM, William Reade william.re...@canonical.com wrote: So, I think you *do* need conversion code specific to state -- the methods exported from state should be in terms of the storage types, because they're the language we've chosen to model storage, so there needs to be conversion code in state (or possibly in an internal package -- but either way there's no reason to expose it to anyone else). Not sure I follow what you're saying about in terms of the storage types, so I'll describe how it is at the moment. The types are in three places: - juju/juju/storage - juju/juju/apiserver/params - juju/juju/state There's a storage.Volume, a params.Volume, and a state.Volume. state.Volume is actually an interface, but there's a VolumeInfo which ~correlates to the other struct types. I think the conversion from params-state all needs to be in apiserver/something. WRT params, though, we already have two groups of client packages, so the same forces don't quite apply: and I still prefer that we *not* import other packages into params [0]. How about a single params/convert package? I could live with it, but I don't like the idea of adding dependencies to every domain for every facade that uses it. Cheers, Andrew Cheers William [0] this is because the temptation to use a storage.Foo in place of a params.Foo has historically been overwhelming (and while it's *very dangerous* to do that it's hard to make the danger sufficiently obvious to dissuade distracted devs from doing so). On Tue, Feb 10, 2015 at 11:07 AM, Andrew Wilkins andrew.wilk...@canonical.com wrote: Hi all, Anastasia raised an issue in http://reviews.vapour.ws/r/885/ about how to cut down on struct conversions between params, state, and domain packages. In this case we're talking about storage. The following API server facades currently participate in storage: - client - storage - provisioner - diskmanager (to be renamed, this lists block devices on machines) - diskformatter - storageworker (to be renamed, this is the dynamic storage provisioner) Each facade have some overlap in dealing with storage entities, e.g. the diskformatter and diskmanager each need to deal with block devices. This leads to much duplication of struct copying/conversion code when toing and froing between state and clients. I don't want to go adding conversion code to the params, state or storage packages, as they really shouldn't have dependencies on each other. Does anyone have a good idea about where to put this common functionality? Maybe api/common/storage, apiserver/common/storage? Does not appeal, but I can't think of a better option. Cheers, Andrew -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: A home for underappreciated errors
On 10 February 2015 at 08:55, Andrew Wilkins andrew.wilk...@canonical.com wrote: Hi all, Ian raised a good point in http://reviews.vapour.ws/r/885/ (caution advised, may cause eyebleed) about a change I made to errors: https://github.com/juju/errors/pull/17 The NotAssigned error, which previously was only raised by state.Unit.AssignedMachineId, is now also raised by state.Volume.StorageInstance. I removed the error from the state package so we didn't have apiserver/common poking its head in there, and moved it over to the juju/errors package. The issue Ian raised is that juju/errors should contain relatively generic error types, and we should keep domain-specific things elsewhere. Examples are NotAssigned, and NotProvisioned. We're thinking of moving these domain-specific error types into a new package, juju/juju/errors. What do you think about this? I agree with Ian here. Personally, I think that specific error types/values work well when defined in or close to the module that returns them, which is aligned to my general feeling that the set of possible returned errors is part of the contract of the method it was returned from. For example, the mgo package returns mgo.ErrNotFound when an item isn't found, and that's fine - we translate from that error to a more local not-found error when necessary. For errors that pass over an API, they can be considered a part of the API and defined along with the other API types. So, rather than a single package for juju-related errors, I would suggest that domain-specific errors are defined in packages close to the domain in question rather than in a single package used by everything. To my mind this is a more modular and scalable approach. It's also the standard Go idiom. cheers, rog. -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: A home for underappreciated errors
I think that NotAssigned is *too* generic. UnitNotAssignedToMachineErr is one thing, and VolumeNotAssignedToStorageInstance is another; if we use the same error type to represent those distinct situations, we really ought to include a bunch more mechanism for finding out *precisely* what should have been assigned to what (because refactoring happens, and methods start returning the same generic error for new reasons, potentially breaking client code that makes assumptions about what that generic error type implies. See worker/uniter/filter/filter.go and search for ErrTerminateAgent for a particularly hair-raising example...) I see what you're saying re apiserver/common, but I wouldn't really characterise it as poking into state: apiserver is generally implemented in terms of state, so that's a perfectly reasonable dependency IMO. For exactly the reasons stated re generic error types in code, the generic error codes in the api are the scary bits I now wish we'd done differently... Cheers William On Tue, Feb 10, 2015 at 10:55 AM, Andrew Wilkins andrew.wilk...@canonical.com wrote: Hi all, Ian raised a good point in http://reviews.vapour.ws/r/885/ (caution advised, may cause eyebleed) about a change I made to errors: https://github.com/juju/errors/pull/17 The NotAssigned error, which previously was only raised by state.Unit.AssignedMachineId, is now also raised by state.Volume.StorageInstance. I removed the error from the state package so we didn't have apiserver/common poking its head in there, and moved it over to the juju/errors package. The issue Ian raised is that juju/errors should contain relatively generic error types, and we should keep domain-specific things elsewhere. Examples are NotAssigned, and NotProvisioned. We're thinking of moving these domain-specific error types into a new package, juju/juju/errors. What do you think about this? Cheers, Andrew -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: A home for underappreciated errors
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10.02.2015 11:57, roger peppe wrote: On 10 February 2015 at 08:55, Andrew Wilkins andrew.wilk...@canonical.com wrote: Hi all, Ian raised a good point in http://reviews.vapour.ws/r/885/ (caution advised, may cause eyebleed) about a change I made to errors: https://github.com/juju/errors/pull/17 The NotAssigned error, which previously was only raised by state.Unit.AssignedMachineId, is now also raised by state.Volume.StorageInstance. I removed the error from the state package so we didn't have apiserver/common poking its head in there, and moved it over to the juju/errors package. The issue Ian raised is that juju/errors should contain relatively generic error types, and we should keep domain-specific things elsewhere. Examples are NotAssigned, and NotProvisioned. We're thinking of moving these domain-specific error types into a new package, juju/juju/errors. What do you think about this? NotAssigned is more generic than NotProvisioned IMO, so if I were to decide which one to move to juju/errors, I'd move the former, not the latter. I agree with Ian here. Personally, I think that specific error types/values work well when defined in or close to the module that returns them, which is aligned to my general feeling that the set of possible returned errors is part of the contract of the method it was returned from. For example, the mgo package returns mgo.ErrNotFound when an item isn't found, and that's fine - we translate from that error to a more local not-found error when necessary. For errors that pass over an API, they can be considered a part of the API and defined along with the other API types. So, rather than a single package for juju-related errors, I would suggest that domain-specific errors are defined in packages close to the domain in question rather than in a single package used by everything. To my mind this is a more modular and scalable approach. +1 I like the idea of having state/errors for sharable-but-domain-specific errors (or network/errors, storage/errors, etc.) and leave generic errors useful it a lot more cases in juju/errors. It's also the standard Go idiom. cheers, rog. - -- Dimiter Naydenov dimiter.nayde...@canonical.com Juju Core Sapphire team http://juju.ubuntu.com -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQEcBAEBAgAGBQJU2d83AAoJENzxV2TbLzHwd20H/i22GBC5dADZnLFAfNhE+X0y E7tZc+UkrsKGUmcAHm1k5tEuv3r9Vk6vLxtEcRVXcWvXqoCCW72Cpsw7lLiAkKoH dYPwyBTVorp/N+Z1hxMN+P3LhctRlGNlz1IR9BLJ+3fM/Dlfp+YfVJ93BfqrcC7f uaUJY0iSdOHuedfs29CtTtamSDNb1s3l8uCo5X0Ihh2zc1SMOHPVFxrdWnLTERpN Vw8C6gv47GETUekTzwoviioiQrDxKijMqjkGB9khrKiXdlSLz0rQV2W+874ztPgo pkcdlSzLCJ89r7WB79kajGYWGrlXD1n6NsTSgywIW+M8teIprHjR/exWDxtyPKs= =dbEQ -END PGP SIGNATURE- -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
[Review Queue] Dell-OMSA, Seafile, Nagios, Quassel-Core
I reviewed the Dell OpenManageServerAdministrator charm which is a brilliant use of a localized juju charm for deploying some added zing to dell servers. https://bugs.launchpad.net/charms/+bug/1325700 This landed today and was +1'd thanks to mwennings hard work. MariaDB had an ISV Maintainer contribution by dbart - who's hard work has been ack'd as the upstream charm https://bugs.launchpad.net/charms/+bug/1028093 There were 2 seafile merges proposed that were accepted today. 1 was for some bug fixups https://code.launchpad.net/~jose/charms/precise/seafile/fixes/+merge/244636 Another was for test inclusion https://code.launchpad.net/~marcoceppi/charms/precise/seafile/tests/+merge/240964 This work was landed due in part to our prior abandoned charms work - and has a bug associated with it: https://bugs.launchpad.net/charms/+source/seafile/+bug/1420374 Nagios template config update: https://code.launchpad.net/~larry-e-works/charms/trusty/nagios/nagios-added-template-config/+merge/242444 This merge was marked as needs work due to missing test coverage and documentation. Minor fixups to bring the submission up to charm store guidelines. -- All the best, Charles Butler charles.but...@canonical.com - Juju Charmer Come see the future of datacenter orchestration: http://jujucharms.com -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
juju-core 1.21.2 is proposed for release
# juju-core 1.21.2 A new stable release of Juju, juju-core 1.21.2, is proposed. This release may replace 1.21.1 on 2015-02-12 after a period of evaluation. ## Getting Juju juju-core 1.21.2 is available for utopic and backported to earlier series in the following PPA: https://launchpad.net/~juju/+archive/proposed The proposed packages in this archive use the proposed simple-streams. You must configure the 'agent-stream' option in your environments.yaml to use the matching juju agents. agent-stream: proposed ## Notable Changes This releases addresses stability and performance issues. ## Resolved issues * Unable to override network-bridge if container type is kvm (local provider) Lp 1416134 * Src/bitbucket.org/kardianos/osext/license is wrong Lp 1416425 * Github.com/juju/syslog has contradicting licensing info Lp 1416433 * Some files refer to an include license file that is not included Lp 1416430 * Github.com/juju/utils has contradictory licence info Lp 1416436 * Juju restore no longer works with azure: error: cannot re-bootstrap environment: cannot determine state server instances: environment is not bootstrapped Lp 1417178 * Unit ports not populated by api megawatcher Lp 1418433 * Apt-proxy can be incorrectly set when the fallback from http-proxy is used Lp 1417617 Finally We encourage everyone to subscribe the mailing list at juju-...@lists.canonical.com, or join us on #juju-dev on freenode. -- Curtis Hovey Canonical Cloud Development and Operations http://launchpad.net/~sinzui -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Unable to deploy a working 10 node hadoop cluster
Ken, Unfortunately, this is due to an upstream change in the openjdk package. A bug has been filed[1] and it has been noted that this breaks all of the Big Data charms in the store. Please indicate on that bug that this is affecting you. There is a symlink work-around mentioned on the bug, but it is per-process. You could also manually install the previous Java version using apt[2], but this is not recommended and would require removing and re-adding all of the relations, at least. [1] https://bugs.launchpad.net/ubuntu/+source/openjdk-6/+bug/1417962 [2] sudo apt-get install openjdk-7-jdk=7u51-2.4.6-1ubuntu4 openjdk-7-jre=7u51-2.4.6-1ubuntu4 openjdk-7-jre-headless=7u51-2.4.6-1ubuntu4 On Tue, Feb 10, 2015 at 11:59 AM, Ken Williams ke...@theasi.co wrote: Hi, I'm trying to deploy a basic hadoop cluster on Amazon (AWS). It should have just 10 nodes to simply run hadoop map-reduce jobs and just store data on hdfs. Nothing else. No hive or spark or anything. These are the commands I enter. juju quickstart juju deploy hdp-hadoop yarn-hdfs-master juju deploy hdp-hadoop compute-node juju add-unit -n 10 compute-node juju add-relation yarn-hdfs-master:namenode compute-node:datanode juju add-relation yarn-hdfs-master:resourcemanager compute-node:nodemanager In 'juju status' I can see all the nodes are being added and I wait until all their statuses are 'running'. If I 'juju ssh' to *any* of the compute-node machines I cannot list any hdfs directories, and get this message root@ip-172-31-28-205:~# su hdfs hdfs@ip-172-31-28-205:/home/ubuntu$ hdfs dfs -ls / ls: Incomplete HDFS URI, no host: hdfs://TODO-NAMENODE-HOSTNAME:PORT hdfs@ip-172-31-28-205:/home/ubuntu$ Also, there is no 'DataNode' process running on the machine, which would be needed to access HDFS. Am I doing something wrong or am I meant to edit the 'hdfs-site.xml' file myself ? On all 10 machines ? Also, if I 'juju ssh' onto the yarn-hdfs-master/0 machine and try to run a hdfsadmin -report, it tells me that hdfs has no data-nodes running (see below) - so when I try to put data on to hdfs it fails with an error message 'There are 0 datanode(s) running'. hdfs@ip-172-31-21-161:/home/ubuntu$ hdfs dfsadmin -report Configured Capacity: 0 (0 B) Present Capacity: 0 (0 B) DFS Remaining: 0 (0 B) DFS Used: 0 (0 B) DFS Used%: NaN% Under replicated blocks: 0 Blocks with corrupt replicas: 0 Missing blocks: 0 - Datanodes available: 0 (0 total, 0 dead) I don't understand if I am doing something wrong. What is the recommended way for deploying a hadoop and hdfs cluster using juju ? Thankyou for any help, Ken -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
What are blocks ?
Hello, Can someone direct me to the design documents for the enigmatic 'block' functionality which is showing up in the review queue. Thanks Dave -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
[Review Queue]: Openbook
For my time in the review queue I did another round on the Talligent Openbook charm. The previous problem was fixed, but it had another configuration related problem. https://bugs.launchpad.net/charms/+bug/1411402 - Matt Bruzek matthew.bru...@canonical.com -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Juju devel 1.22-beta3 is available for testing.
# juju-core 1.22-beta3 A new development release of Juju, juju-core 1.22-beta3, is now available. This release replaces 1.22-beta2. ## Getting Juju juju-core 1.22-beta3 is available for utopic and backported to earlier series in the following PPA: https://launchpad.net/~juju/+archive/devel The devel packages in this archive use the devel simple-streams. You must configure the 'agent-stream' option in your environments.yaml to use the matching juju agents. agent-stream: devel Upgrading from stable releases to development releases is not supported. You can upgrade test environments to development releases to test new features and fixes, but it is not advised to upgrade production environments to 1.22-beta3. ## Notable Changes This releases addresses stability and performance issues. ## Resolved issues * Failure to retrieve the template to clone: lxc container with 1.22 beta2 Lp 1417594 * Charm download behind the enterprise proxy fails Lp 1403225 * Unit ports not populated by api megawatcher Lp 1418433 * Apt-proxy can be incorrectly set when the fallback from http-proxy is used Lp 1417617 Finally We encourage everyone to subscribe the mailing list at juju-...@lists.canonical.com, or join us on #juju-dev on freenode. -- Curtis Hovey Canonical Cloud Development and Operations http://launchpad.net/~sinzui -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
