[mcollective-users] Choria releases
Hello, I am pleased to announce a few quite significant Choria releases. The focus of these releases is security: You can now integrate the CLI with your enterprise SSO and entitlement system which could include centralised RBAC and Auditing. I will publish a blog post covering this in depth and also soon a open source AAA system that support this. Keep an eye on the list for the blog post or subscribe to our blog[1]. The use of the Choria Client can be limited to specific IP addresses, I blogged about Limiting Clients to IP Ranges[2]. Thanks to Jeroen Schutrup for his contribution. *Choria Server 0.10.0:* * Various fixes to privileged security certificate handling via *go- security* release *0.3.0* * Allow limiting clients to sets of IPs via *go-network-broker#12* * Ensure the server status file is world readable * Force exit even when worker routines are not done after *soft_shutdown_timeout*, default 2 seconds * Further fixes to avoid concurrent hash access panics for golang client code * Include the server version when creating life cycle events * Improve *alive* event spread by sleeping for up to a hour for initial publish * Expose *security.Validate* to users of the go framework *choria/mcollective_choria and choria/mcollective_agent_bolt_tasks 0.13.0:* * Support integrating Choria CLI with centralised AAA services *choria/mcollective_agent_nrpe 4.1.0:* * Support running commands as non root via sudo -- R.I.Pienaar / www.devco.net / @ripienaar Links: 1. https://choria.io/blog/ 2. https://choria.io/blog/post/2019/01/17/client_ip_limits/ -- --- You received this message because you are subscribed to the Google Groups "mcollective-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to mcollective-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[mcollective-users] Choria Releases
hello, I am pleased to announce releases from the Choria project. Notable achievements are that *mco facts* will once again work when used with the Choria Server, while this in itself is a good thing what this implies is that a whole class of compatibility issues when moving from *mcollectived* to *Choria Server* have been fixed, those who wrote weird custom agents in the past will have a smoother migration now. The upcoming Choria Server release fully support intermediate certificate chains and non Puppet CAs. The Choria Provisioner supports enrolling Server into custom CAs and we'll have documentation up for a CFSSL based CA. In these releases we support setting custom (non Puppet) SSL related paths in the configuration files for the Ruby daemon and client. I'll release a Choria Server in the next few days, you might want to wait for that. Huge shout out to Vincent Janelle for all his help on the SSL side of things! *choria/mcollective_choria and choria/mcollective_agent_bolt_tasks version 0.12.1:* * Improve backward compatibility in JSON mode * Support disabling PKI and TLS separately * Support identical settings to the go file security mode allowing non Puppet SSL file paths * Support intermediate certificate chains * Export the PKI and TLS modes via choria_util#info * Increase choria_util timeout to allow for slow facter -- R.I.Pienaar / www.devco.net / @ripienaar -- --- You received this message because you are subscribed to the Google Groups "mcollective-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to mcollective-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [mcollective-users] Choria Releases
hello, Further to this I just released choria/choria 0.7.1 which resolved 2 debian issues: * apt-get update is run when adding repos * identity is now specifically configured to the fqdn Thanks Jason Spalding and Mateusz Gozdek for your kind help On Wed, 21 Mar 2018, at 16:53, R.I.Pienaar wrote: > Hello, > > I am very please to announce a number of releases from the Choria > Project. > This is a major release that delivers on a number of the roadmap items I > mentioned recently and that will unfortunately require some > reconfiguration of your site. A number of items are being deprecated > for newer, faster, lighter and easier to use versions of the same. > *Major New Features:* > > * There is now a Puppet Tasks runner that does not rely on SSH, does >not need Bolt and have strong RBAC and Auditing > * We now have a new Choria Network Broker that replace the NATS broker > * We now have a Choria Federation Broker that replace the previous Ruby >Federation Broker > * Choria Data Adapters[1] a step towards stream processing and building >very large scale node metadata ingest networks > *Deprecations:* > > * The NATS module and the NATS server deployment method > * The old Ruby based Federation Broker > *Removed:* > * YAML Playbooks > * The *mco federation observe* command > > This all seems like a huge change, but in reality moving to the new > Choria Broker is really easy and the resulting build is more robust, use > fewer resources and scales further. Please review the Network Broker > documentation[2]. RPMs and Debs are supplied and some lovely > contributors are working on getting it into Archlinux base distro. > Puppet Tasks are a major new capability that along with the Puppet based > Playbooks from the previous release rounds out the major road map items > I had for delivering cutting edge enterprise level features to the > Puppet Ecosystem as Open Source. As such they are fast, stable, > consistent and secure. Please review the Puppet Tasks documentation[3]. > Please pay special attention to the Status section. > Network Federation has had a big overhaul - the new Federation Brokers > are incredibly performant and scalable while using significantly > fewerresources and being easier to deploy. I've tested these to ~ 15 > 000 federated networks. Documentation has been updated[4]. > While on the topic of scale I would like to point out a blog post I > recently wrote about my first real world 50 000 node Choria deploy[5]. > This deployment is using the Network Broker and other components > released today. Watch this space for a several 100% increase in node > counts and details about that. > > *choria/nats version 0.4.0* > * This module is now deprecated and a easy *ensure => absent* flag has >been added to clean up after it > *choria/choria version 0.7.0* > > * Support managing the Choria Network Broker > * Support managing the Choria Federation Broker > * Support managing the Choria Data Adapters > * Support managing Ubuntu and Debian repositories > * Improve internal module ordering > * Add mandatory name option to the YUM repositories > *choria/mcollective_choria version 0.7.0* > > * When running playbooks default to the users module path > * Update the NATS client gem to 0.4.0 > * Support Puppet Tasks > * Support multiple module paths on the playbook CLI > * Fix dependency handling on Archlinux > * Deprecate the Ruby Federation Broker > * Remove YAML playbook > * Remove the mco federation observe command > -- > R.I.Pienaar / www.devco.net / @ripienaar > > Links: > > 1. https://master.choria.io/docs/adapters/ > 2. https://choria.io/docs/deployment/broker/ > 3. https://choria.io/docs/tasks/ > 4. https://choria.io/docs/federation/ > 5. https://www.devco.net/archives/2018/03/07/50-000-choria-node-network.php > > -- > > --- > You received this message because you are subscribed to the Google > Groups "mcollective-users" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to mcollective-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- R.I.Pienaar / www.devco.net / @ripienaar -- --- You received this message because you are subscribed to the Google Groups "mcollective-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to mcollective-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[mcollective-users] Choria Releases
Hello, I am very please to announce a number of releases from the Choria Project. This is a major release that delivers on a number of the roadmap items I mentioned recently and that will unfortunately require some reconfiguration of your site. A number of items are being deprecated for newer, faster, lighter and easier to use versions of the same. *Major New Features:* * There is now a Puppet Tasks runner that does not rely on SSH, does not need Bolt and have strong RBAC and Auditing * We now have a new Choria Network Broker that replace the NATS broker * We now have a Choria Federation Broker that replace the previous Ruby Federation Broker * Choria Data Adapters[1] a step towards stream processing and building very large scale node metadata ingest networks *Deprecations:* * The NATS module and the NATS server deployment method * The old Ruby based Federation Broker *Removed:* * YAML Playbooks * The *mco federation observe* command This all seems like a huge change, but in reality moving to the new Choria Broker is really easy and the resulting build is more robust, use fewer resources and scales further. Please review the Network Broker documentation[2]. RPMs and Debs are supplied and some lovely contributors are working on getting it into Archlinux base distro. Puppet Tasks are a major new capability that along with the Puppet based Playbooks from the previous release rounds out the major road map items I had for delivering cutting edge enterprise level features to the Puppet Ecosystem as Open Source. As such they are fast, stable, consistent and secure. Please review the Puppet Tasks documentation[3]. Please pay special attention to the Status section. Network Federation has had a big overhaul - the new Federation Brokers are incredibly performant and scalable while using significantly fewerresources and being easier to deploy. I've tested these to ~ 15 000 federated networks. Documentation has been updated[4]. While on the topic of scale I would like to point out a blog post I recently wrote about my first real world 50 000 node Choria deploy[5]. This deployment is using the Network Broker and other components released today. Watch this space for a several 100% increase in node counts and details about that. *choria/nats version 0.4.0* * This module is now deprecated and a easy *ensure => absent* flag has been added to clean up after it *choria/choria version 0.7.0* * Support managing the Choria Network Broker * Support managing the Choria Federation Broker * Support managing the Choria Data Adapters * Support managing Ubuntu and Debian repositories * Improve internal module ordering * Add mandatory name option to the YUM repositories *choria/mcollective_choria version 0.7.0* * When running playbooks default to the users module path * Update the NATS client gem to 0.4.0 * Support Puppet Tasks * Support multiple module paths on the playbook CLI * Fix dependency handling on Archlinux * Deprecate the Ruby Federation Broker * Remove YAML playbook * Remove the mco federation observe command -- R.I.Pienaar / www.devco.net / @ripienaar Links: 1. https://master.choria.io/docs/adapters/ 2. https://choria.io/docs/deployment/broker/ 3. https://choria.io/docs/tasks/ 4. https://choria.io/docs/federation/ 5. https://www.devco.net/archives/2018/03/07/50-000-choria-node-network.php -- --- You received this message because you are subscribed to the Google Groups "mcollective-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to mcollective-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[mcollective-users] Choria Releases
Hello, I released a few updates today, these are either features or bug fixes and one deprecation. Special thanks to Luke Bigum for contributing to these releases A reminder to subscribe to the choria-users mailing list[1] to stay up to date with developments *choria/mcollective_choria 0.1.0:** * *Enhancements:** * * Add *assert* to the mcollective playbook task using JGrep * Support facter dot syntax for structured facts in fact filters when using the choria discovery method * Raise a error when the client is run as root rather than fail silently * In the config files server lists that are comma separated can now have spaces between them *Bug Fixes:** * * Correctly parse request statuses when mcollective is in JSON pure mode in playbooks * Remove the dependency on the supervisor module which hampered installing Choria along with modern concat * Fix fact filters in JSON pure mcollectives *Deprecations:** * * The *mcollective_assert* playbook task is now deprecated, use *assert* in the *mcollective* task. It still work but will be removed soon *choria/mcollective 0.0.29:** * *Enhancements:* * Specifying *--vendor* when packaging plugins is required, a nicer error will be produced now *choria/discovery_proxy 0.1.0:** * *Enhancements:** * * To match functionality with choria this also now supports the dot notation for fact queries -- R.I.Pienaar / www.devco.net / @ripienaar Links: 1. https://groups.google.com/forum/#!forum/choria-users -- --- You received this message because you are subscribed to the Google Groups "mcollective-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to mcollective-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[mcollective-users] Choria releases headsup
hello, It has been a while since the last Choria releases, the reasons for this is that I was stabilising the network protocols before releasing a new Go based broker and federation broker package. I wanted to give you a headsup that the next Choria release will have 2 gotchas. If you are just blindly updating to these modules please keep in mind! For the upcoming Go based components to work correctly I had to make a small change to the networking protocol of Choria and having made it I am versioning those as version 1 of the protocol. The end result is unfortunately that updating to the next Choria release will introduce a incompatibility between old and new. Your old client can communicate to new servers but not the other way round. Consider this before upgrading. It's unfortunate this had to happen but that's why I never made Choria 1.0.0, we're one step closer to that and the stability it implies. Another change that will land at the same time is related to config files. As of right now any config already in your server/client.cfg files will be left alone and only some settings changed. This has make installation a pain because I had to insist on factory default configs and also made recovering from mistakes problematic. Going forward the entire config is managed, nothing you put into the cfg files outside of the choria/mcollective module will survive, including comments and ordering of the config file. I anticipate these releases will land in the next few days, maybe next week. I am also looking for Beta testers for a new network broker and federation broker all in one binary if anyone feel like spending some of their time testing these with me get in touch. -- R.I.Pienaar / www.devco.net / @ripienaar -- --- You received this message because you are subscribed to the Google Groups "mcollective-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to mcollective-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[mcollective-users] Choria Releases
Hello, I am pleased to announce a bunch of releases from the Choria project. *Announcing Choria Discovery Proxy:* In this batch of releases comes a new project called the Choria Discovery Proxy. It's a proxy service in front of PuppetDB that takes care of doing discovery for MCollective. This deals with the problem where PuppetDB based discovery required all clients to be able to do PQL queries in PuppetDB which would be a big potential security concern. With this proxy the only information that can be extracted from PuppetDB is certificate names. It also lets you store PQL queries by name and later reference those in discovery for example *-I set:acme_servers* This is the first Golang based project that forms part of Choria and already includes a full configuration parser for MCollective, SRV support and more. It's a precursor to a future project to replace the MCollective daemon with a much lighter daemon. *choria/mcollective_choria release 0.0.27:* *New Features:* * Improved Certname validation to ensure the SSL setup is sane * Add a etcd data store for Playbooks * Support the new Choria Discovery Proxy * Update to Hiera 5 and require Puppet > 4.9.0 *Bug Fixes:* * Fix federation trace from a client to the server on the same node *Preview Feature:* MCollective have always been based on a YAML based transport which transported some Ruby specific data. This was a huge blocker to supporting REST gateways and other programming languages. In an upcoming MCollective release a translation layer exists to allow a JSON pure transport that uses only JSON primitive data types. This release of Choria lets you enable this JSON pure transport using *plugin.choria.security.serializer = json* setting. Note though this is marked as preview as I will probably formalise the network protocol a bit and things might break so only test this if you feel adventurous. *choria/mcollective release 0.0.27:* *New Features:* * Support local gem mirrors * Improve the README template for generated modules * Update to Hiera 5 and require Puppet > 4.9.0 *choria/nats release 0.0.11:* *Bug Fixes:* * Handle passwords with special characters better * Notify instead of fail on missing Hiera data * Update to Hiera 5 and require Puppet > 4.9.0 *choria/discovery_proxy release 0.0.1:* *Initial Release* Massive thanks to Jos Houtman and Mateusz Gozdek for all their help with these releases -- R.I.Pienaar / www.devco.net / @ripienaar -- --- You received this message because you are subscribed to the Google Groups "mcollective-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to mcollective-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
