Re: [Puppet-dev] Re: Thoughts on Module Namespace Conflicts
I'll resurrect this old thread to give an update. We landed a function
in stdlib to do some of what I discussed in my last post, by inspecting
module metadata directly.
The function:
https://github.com/puppetlabs/puppetlabs-stdlib#load_module_metadata
A blog post I wrote about some possible uses:
http://scienceofficersblog.blogspot.com/2015/08/inspecting-puppet-module-metadata.html
--
Spencer Krum
n...@spencerkrum.com
On Mon, Feb 23, 2015, at 03:32 PM, Spencer Krum wrote:
The use case I think for having multiple modules of the same name
available is quite limited. I don't think anyone would claim best
practice if they had two apache modules in their modulepath.
I also think there is another component which is versioning. Right now
there is no system for a profile module to know what version of e.g.
rabbitmq is installed. This means it just assumes one and fails if the
rabbitmq class takes different parameters now. One terrible workaround
for this is for classes to accept a 'rabbitmq_module_version' parameter
and then switch on it inside the class. This workaround is actually
used.
So I think as long as we're discussing how to make module owner
detectable, requireable, whatever, that we should involve versioning as
well, because bumping minor or major versions of utility modules like
postgres, mysql, rabbitmq happens a lot.
One case that actually happens where there is a want for two different
modules by the same name, is when an organization is migrating. An org
that is moving from example42-apache to puppetlabs-apache isn't gonna
want to have a flag day where everything is broken. They don't really
want to play move hosts one by one out of an environment into a testing
environment then into a 'done' environment. This is where being able to
have two modules of the same name could be really useful.
Erik proposed allowing modules to specify an interface, then other
modules could implement that interface. I don't think that is a likely
outcome. I think when we have two modules that both manage 'apache' the
only reason one hasn't totally taken over the other is because they have
different underlying ideas about how to manage apache. I.e. what the
inputs are and what the scope of the outputs are.
We on the openstack infrastructure team have even gone so far as to fork
an old version of the apache module to openstackinfra-httpd. This is
0.0.4 of the pl-apache module and takes wildly different inputs. There
are three reasons for the fork. One, this module takes a template, fills
it out, and dumps it in a vhost. Clean and simple. The way we think it
should be. Two, now that we have forked we can update the codebase,
clean it up, and fix bugs. Three, with the module now living as 'httpd'
in our module path, we, or others consuming our infra, can use a modern
version of the apache module for whatever they need.
As a final thought. One way to deal with this problem is to socialize
the requirement of a new metadata file (yay) called something like
manifests/properties.pp. This file would have all the data in
metadata.json. But since it is readable by puppet we could end up with
code that looks like this in our profiles:
class profile () {
case $::apache::properties::author {
'puppetlabs': {
// do some apache stuff the puppetlabs way
}
'example42': {
// do some apache stuff the example42 way
}
'default': { fail(please stop writing your own apache module) }
}
}
And the technique for handling multiple versions of the same module is
basically exactly the same. The cool part of this is that it requires no
code changes to puppet core. The bad part is we'd have to socialize it
and thats really hard.
--
Spencer Krum
n...@spencerkrum.com
On Fri, Feb 20, 2015, at 04:34 PM, Henrik Lindberg wrote:
On 2015-20-02 20:51, Wil Cooley wrote:
In some ways, this is a lot like environments, but unlike environments,
which are unique at the node-level, these allow the use of multiple
module-sets and are scoping for entities at the manifest-level. As I
understand environments, they can (and probably usually should) be
relatively isolated from each other; the purpose of namespaced
module-sets is to allow addressing without isolation. Maybe this is what
Henrik's last message in this thread is getting at when he says:
I have claimed on many occasions that an environment is just like a
module (with higher privileges / precedence). The set of available
modules should be more of a repository kind of thing that the loader
resolves modules from.
I meant that an environment can be seen as a module. Instead of having a
modulepath, it should list all the modules that it requires (those
modules in turn can require other modules). When a module is needed it
should be resolved against a repository of modules. Such a repository
Re: [Puppet-dev] Re: Thoughts on Module Namespace Conflicts
The use case I think for having multiple modules of the same name
available is quite limited. I don't think anyone would claim best
practice if they had two apache modules in their modulepath.
I also think there is another component which is versioning. Right now
there is no system for a profile module to know what version of e.g.
rabbitmq is installed. This means it just assumes one and fails if the
rabbitmq class takes different parameters now. One terrible workaround
for this is for classes to accept a 'rabbitmq_module_version' parameter
and then switch on it inside the class. This workaround is actually
used.
So I think as long as we're discussing how to make module owner
detectable, requireable, whatever, that we should involve versioning as
well, because bumping minor or major versions of utility modules like
postgres, mysql, rabbitmq happens a lot.
One case that actually happens where there is a want for two different
modules by the same name, is when an organization is migrating. An org
that is moving from example42-apache to puppetlabs-apache isn't gonna
want to have a flag day where everything is broken. They don't really
want to play move hosts one by one out of an environment into a testing
environment then into a 'done' environment. This is where being able to
have two modules of the same name could be really useful.
Erik proposed allowing modules to specify an interface, then other
modules could implement that interface. I don't think that is a likely
outcome. I think when we have two modules that both manage 'apache' the
only reason one hasn't totally taken over the other is because they have
different underlying ideas about how to manage apache. I.e. what the
inputs are and what the scope of the outputs are.
We on the openstack infrastructure team have even gone so far as to fork
an old version of the apache module to openstackinfra-httpd. This is
0.0.4 of the pl-apache module and takes wildly different inputs. There
are three reasons for the fork. One, this module takes a template, fills
it out, and dumps it in a vhost. Clean and simple. The way we think it
should be. Two, now that we have forked we can update the codebase,
clean it up, and fix bugs. Three, with the module now living as 'httpd'
in our module path, we, or others consuming our infra, can use a modern
version of the apache module for whatever they need.
As a final thought. One way to deal with this problem is to socialize
the requirement of a new metadata file (yay) called something like
manifests/properties.pp. This file would have all the data in
metadata.json. But since it is readable by puppet we could end up with
code that looks like this in our profiles:
class profile () {
case $::apache::properties::author {
'puppetlabs': {
// do some apache stuff the puppetlabs way
}
'example42': {
// do some apache stuff the example42 way
}
'default': { fail(please stop writing your own apache module) }
}
}
And the technique for handling multiple versions of the same module is
basically exactly the same. The cool part of this is that it requires no
code changes to puppet core. The bad part is we'd have to socialize it
and thats really hard.
--
Spencer Krum
n...@spencerkrum.com
On Fri, Feb 20, 2015, at 04:34 PM, Henrik Lindberg wrote:
On 2015-20-02 20:51, Wil Cooley wrote:
In some ways, this is a lot like environments, but unlike environments,
which are unique at the node-level, these allow the use of multiple
module-sets and are scoping for entities at the manifest-level. As I
understand environments, they can (and probably usually should) be
relatively isolated from each other; the purpose of namespaced
module-sets is to allow addressing without isolation. Maybe this is what
Henrik's last message in this thread is getting at when he says:
I have claimed on many occasions that an environment is just like a
module (with higher privileges / precedence). The set of available
modules should be more of a repository kind of thing that the loader
resolves modules from.
I meant that an environment can be seen as a module. Instead of having a
modulepath, it should list all the modules that it requires (those
modules in turn can require other modules). When a module is needed it
should be resolved against a repository of modules. Such a repository
could be implemented many ways; one being that it is just the modules
now on the modulepath.
The main difference is that the resolution from name to actual module is
now free from implementation concerns.
Currently the dependencies that are expressed are between the containers
of logic (the module depends on another specific module with a specific
implementations e.g. puppetlabs/stdlib). This is simple but not very
flexible. It would be far better if the requirements were established
based on what is actually required from that module - say that it needs
a
[Puppet-dev] Re: Thoughts on Module Namespace Conflicts
On 2015-20-02 20:51, Wil Cooley wrote: In some ways, this is a lot like environments, but unlike environments, which are unique at the node-level, these allow the use of multiple module-sets and are scoping for entities at the manifest-level. As I understand environments, they can (and probably usually should) be relatively isolated from each other; the purpose of namespaced module-sets is to allow addressing without isolation. Maybe this is what Henrik's last message in this thread is getting at when he says: I have claimed on many occasions that an environment is just like a module (with higher privileges / precedence). The set of available modules should be more of a repository kind of thing that the loader resolves modules from. I meant that an environment can be seen as a module. Instead of having a modulepath, it should list all the modules that it requires (those modules in turn can require other modules). When a module is needed it should be resolved against a repository of modules. Such a repository could be implemented many ways; one being that it is just the modules now on the modulepath. The main difference is that the resolution from name to actual module is now free from implementation concerns. Currently the dependencies that are expressed are between the containers of logic (the module depends on another specific module with a specific implementations e.g. puppetlabs/stdlib). This is simple but not very flexible. It would be far better if the requirements were established based on what is actually required from that module - say that it needs a function 'foo'. Then that could be declared and any module that happens to make that function available is a potential replacement. The same is applicable to other elements of the puppet language, classes, defines, types, etc. Such a scheme requires something to select the modules that are actually wanted out of all potential modules. This could be the role of the environment module - it would have dependencies on the modules that should be used. It is not limited to that however since any module can act as a configuration (by having explicit dependencies on other modules) - such modules does not have to contain anything but the dependencies. They become reusable parts of configuration thus making it easier to set up many different environments. While this increases flexibility it also adds complexity. It is also not the complete solution to all the things discussed in this thread, but I believe it could be the foundation for most of them. - henrik -- Visit my Blog Puppet on the Edge http://puppet-on-the-edge.blogspot.se/ -- You received this message because you are subscribed to the Google Groups Puppet Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/mc8jqc%24hus%241%40ger.gmane.org. For more options, visit https://groups.google.com/d/optout.
[Puppet-dev] Re: Thoughts on Module Namespace Conflicts
On 2015-12-02 15:12, Erik Dalén wrote: A related ticket to the interface and redirect functionality is https://tickets.puppetlabs.com/browse/FORGE-111 There I just suggest the feature to be able to say that module B provides the interface of module A version X. So either module can satisfy a dependency on module A version X. Then if you want to have some sort of generic interface that other modules can implement, you could have a sort of virtual module with he version being the interface version. And then other modules provide or depend on that virtual module. The meaning of virtual module here is the same as for virtual packages in package managers, like ruby-interpreter or smtp-server in Debian, and the implementation would also be pretty similar. Regarding having multiple variants or versions of a module being installed at the same time I think we should change the environment loader API to provide a list of modules instead of a list of module paths. Then plugins could implement any sort of behaviour for this without jumping through hoops. I have claimed on many occasions that an environment is just like a module (with higher privileges / precedence). The set of available modules should be more of a repository kind of thing that the loader resolves modules from. Essentially this is how the new loaders work (used only for 4x functions so far). - henrik -- Visit my Blog Puppet on the Edge http://puppet-on-the-edge.blogspot.se/ -- You received this message because you are subscribed to the Google Groups Puppet Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/mbii8r%24mae%241%40ger.gmane.org. For more options, visit https://groups.google.com/d/optout.
[Puppet-dev] Re: Thoughts on Module Namespace Conflicts
On 2015-10-02 1:23, Trevor Vaughan wrote:
I was talking with a few folks today about potential resolutions to
module namespace issues.
== Fundamental Issue ==
puppetlabs_apache -- Installs To -- apache
example42_apache -- Installs To -- apache
theforeman_apache -- Installs To -- apache
You get my point...
== Current Solutions ==
Right now, there are two ways to solve this problem if you want some of
your nodes to use puppetlabs_apache and others to use theforeman_apache.
1) Modify the module and force the namespace:
- puppetlabs_apache -- Installs To -- puppetlabs_apache
- theforeman_apache -- Installs To -- theforeman_apache
* Isssue: You can't just use outside code at will. You have to modify
*everything* that uses the above code.
2) You have to create a separate environment for each host group
(potentially each host) that you want to have use the differing code.
* Issue: This is a LOT of overhead for a couple of modules and may
have other ramifications in terms of performance as you get up in number.
So, would it be possible to allow multiple versions of a module to exist
in the same namespace but only use one during a given run?
Basically no, this is not possible without adding some kind of mechanism
to filter out modules on the modulepath. This will need to be done per
environment anyway. Remember that modules can contribute all sorts of
extensions to puppet (faces, indirections, facts, functions, types, and
with future parser also bindings). For performance reasons (and
bootstrapping, etc.) these are scanned when the environment loads (some
scans are lazy, but may take place before manifests really starts to be
evaluated).
- henrik
== The First Proposal ==
Inspired by RVM Gemsets, how about allowing modules to declare which
version of a given module they will use?
/etc/puppet/modules/apache/puppetlabs/{files,modules,manifests}
/etc/puppet/modules/apache/theforeman/{files,modules,manifests}
/etc/puppet/modules/apache/author/{files,modules,manifests}
Then, use could be dictated by something like: include apache@puppetlabs.
Unfortunately, this really comes down to the parser being able to
understand the following:
include apache = See if there are any @'s and use that one
include apache@puppetlabs = include the Puppetlabs apache
include apache@puppetlabs include apache@theforeman = Fail, conflict
== Alternative ==
One possible alternative is to just use the metadata.json file to
dictate which module will be used when loading other modules.
Again, if there is a conflict, that is a failure but *only* if the code
attempts to use both at the same time.
The benefit here is that it should make things very unambiguous while
the drawback (if it really is) is that you *must* put a metadata.json
file in every module that you create.
This is just a set of thoughts that I hope get us moving in a direction
where this type of thing is possible and I look forward to hearing what
people think (good, bad, or ugly)!
Thanks,
Trevor
--
Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699
tvaug...@onyxpoint.com mailto:tvaug...@onyxpoint.com
-- This account not approved for unencrypted proprietary information --
--
You received this message because you are subscribed to the Google
Groups Puppet Developers group.
To unsubscribe from this group and stop receiving emails from it, send
an email to puppet-dev+unsubscr...@googlegroups.com
mailto:puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoU2unExPve9ig%3DinW9obDpwii7gBi6W4RkHycViibJp-g%40mail.gmail.com
https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoU2unExPve9ig%3DinW9obDpwii7gBi6W4RkHycViibJp-g%40mail.gmail.com?utm_medium=emailutm_source=footer.
For more options, visit https://groups.google.com/d/optout.
--
Visit my Blog Puppet on the Edge
http://puppet-on-the-edge.blogspot.se/
--
You received this message because you are subscribed to the Google Groups Puppet
Developers group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-dev/mbdd0b%24epb%241%40ger.gmane.org.
For more options, visit https://groups.google.com/d/optout.
[Puppet-dev] Re: Thoughts on Module Namespace Conflicts
I'm personally more of a fan of what some programming languages call
interfaces; a sort of contract if you will that a module implements.
So for instance olindata::galera could look for any class that implements
the IMysql interface. That way, I don't really need to worry if the
end-user is using puppetlabs::mysql or example42::mysql, as long as they
implement the IMysql interface I can guarantee it does what I need.
This would have pretty heavy impact on how you write code right now, but it
would be a rather elegant solution imho.
interface IMysql::server identified by
'9ff3d80b-b02d-4994-b4da-e1ff205304ea' {
Service['mysql']
Package['mysql-server']
File['apache2.conf']
}
class puppetlabs::mysql implements
IMysql['9ff3d80b-b02d-4994-b4da-e1ff205304ea'] {
[.. SNIP..]
# file, service, package here like normal
}
class example42::mysql implements
IMysql['9ff3d80b-b02d-4994-b4da-e1ff205304ea'] {
[.. SNIP..]
# file, service, package here like normal
}
class olindata::galera {
package { 'galera':
require = Package['IMysql::server::mysql-server']
}
}
Or something along those lines. Question remains where the IMysql::server
interface then needs to be declared, and how we would manage the GUID
registration of interfaces.
Comments, thoughts?
Walter
On Tuesday, February 10, 2015 at 5:52:37 PM UTC+1, henrik lindberg wrote:
On 2015-10-02 1:23, Trevor Vaughan wrote:
I was talking with a few folks today about potential resolutions to
module namespace issues.
== Fundamental Issue ==
puppetlabs_apache -- Installs To -- apache
example42_apache -- Installs To -- apache
theforeman_apache -- Installs To -- apache
You get my point...
== Current Solutions ==
Right now, there are two ways to solve this problem if you want some of
your nodes to use puppetlabs_apache and others to use theforeman_apache.
1) Modify the module and force the namespace:
- puppetlabs_apache -- Installs To -- puppetlabs_apache
- theforeman_apache -- Installs To -- theforeman_apache
* Isssue: You can't just use outside code at will. You have to modify
*everything* that uses the above code.
2) You have to create a separate environment for each host group
(potentially each host) that you want to have use the differing code.
* Issue: This is a LOT of overhead for a couple of modules and may
have other ramifications in terms of performance as you get up in
number.
So, would it be possible to allow multiple versions of a module to exist
in the same namespace but only use one during a given run?
Basically no, this is not possible without adding some kind of mechanism
to filter out modules on the modulepath. This will need to be done per
environment anyway. Remember that modules can contribute all sorts of
extensions to puppet (faces, indirections, facts, functions, types, and
with future parser also bindings). For performance reasons (and
bootstrapping, etc.) these are scanned when the environment loads (some
scans are lazy, but may take place before manifests really starts to be
evaluated).
- henrik
== The First Proposal ==
Inspired by RVM Gemsets, how about allowing modules to declare which
version of a given module they will use?
/etc/puppet/modules/apache/puppetlabs/{files,modules,manifests}
/etc/puppet/modules/apache/theforeman/{files,modules,manifests}
/etc/puppet/modules/apache/author/{files,modules,manifests}
Then, use could be dictated by something like: include
apache@puppetlabs.
Unfortunately, this really comes down to the parser being able to
understand the following:
include apache = See if there are any @'s and use that one
include apache@puppetlabs = include the Puppetlabs apache
include apache@puppetlabs include apache@theforeman = Fail, conflict
== Alternative ==
One possible alternative is to just use the metadata.json file to
dictate which module will be used when loading other modules.
Again, if there is a conflict, that is a failure but *only* if the code
attempts to use both at the same time.
The benefit here is that it should make things very unambiguous while
the drawback (if it really is) is that you *must* put a metadata.json
file in every module that you create.
This is just a set of thoughts that I hope get us moving in a direction
where this type of thing is possible and I look forward to hearing what
people think (good, bad, or ugly)!
Thanks,
Trevor
--
Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699
tvau...@onyxpoint.com javascript: mailto:tvau...@onyxpoint.com
javascript:
-- This account not approved for unencrypted proprietary information --
--
You received this message because you are subscribed to the Google
Groups Puppet Developers group.
To unsubscribe from this group and stop
Re: [Puppet-dev] Re: Thoughts on Module Namespace Conflicts
This reminds me of ARM-17 that I tried to model after Haskell/Python/Clojure https://github.com/puppetlabs/armatures/pull/64 . It stalled out due to lack of time, though it may have some relevant content. -Hunter On Tue, Feb 10, 2015 at 12:44 PM, Heck, Walter walterh...@olindata.com wrote: Hi Henrik, On Tue, Feb 10, 2015 at 9:30 PM, Henrik Lindberg henrik.lindb...@cloudsmith.com wrote: On 2015-10-02 19:52, Walter Heck wrote: I'm personally more of a fan of what some programming languages call interfaces; a sort of contract if you will that a module implements. Yes that is good - does not solve the issue of modules having the same name though - either the module name must change, or modules must be installed with both author and name and then referenced the same way. But with this you wouldn't need to have more then one module with the same name? All you'd need is a single module that satisfies all the interfaces the rest of your code asks for. I would think that if this was implemented, the community would gravitate towards a single IMysql interface which can then be implemented by different mysql modules in different ways. cheers, -- Best regards, Walter Heck CEO / Founder OlinData https://t.yesware.com/tl/2627942011ccc3835f76c6e9ba4c0af91f3d3722/53f65fbbd041f0e792b2eaca63996aba/53e6d075ec31fd4e14e81c226c7f32cd?ytl=http%3A%2F%2Folindata.com%2F%3Fsrc%3Dwh_gapp - Open Source Training Consulting Check out our upcoming trainings https://t.yesware.com/tl/2627942011ccc3835f76c6e9ba4c0af91f3d3722/53f65fbbd041f0e792b2eaca63996aba/50b3fd816e9110607700dc6c8a41a2cb?ytl=http%3A%2F%2Folindata.com%2Ftraining%2Fupcoming -- You received this message because you are subscribed to the Google Groups Puppet Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CAOfYMj4uk2OUtQ5tbG6waX0K4FQRfUST8Q2hf-sfp5Pz%3DsADJw%40mail.gmail.com https://groups.google.com/d/msgid/puppet-dev/CAOfYMj4uk2OUtQ5tbG6waX0K4FQRfUST8Q2hf-sfp5Pz%3DsADJw%40mail.gmail.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Puppet Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CAJaQvGBtx1P1hvoaS9ySyX9kh97axybQR2t2qOV%2BF6vwKT5CSQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: [Puppet-dev] Re: Thoughts on Module Namespace Conflicts
Hi Trevor, On Tue, Feb 10, 2015 at 9:17 PM, Trevor Vaughan tvaug...@onyxpoint.com wrote: Walter: This would be nice, but variables are the...uh...variable here. Many people only expose the variables that they need while others expose every variable they can find. It would be an interesting experiment to see if it would work though. Well, in for instance Delphi interfaces could also have properties, so I don't think that needs to be a problem. The interface wouldn't determine the value of the property though, just that it should exist in a class implementing that interface. See for some Delphi examples here: http://www.delphibasics.co.uk/Article.asp?Name=Interface -- Best regards, Walter Heck CEO / Founder OlinData https://t.yesware.com/tl/2627942011ccc3835f76c6e9ba4c0af91f3d3722/d6849ea137740bcb1bde36decd89ddfc/74ed25adcab9cd7b409f6599195236ad?ytl=http%3A%2F%2Folindata.com%2F%3Fsrc%3Dwh_gapp - Open Source Training Consulting Check out our upcoming trainings https://t.yesware.com/tl/2627942011ccc3835f76c6e9ba4c0af91f3d3722/d6849ea137740bcb1bde36decd89ddfc/3c18bba479fce00720c14e4a40def5e6?ytl=http%3A%2F%2Folindata.com%2Ftraining%2Fupcoming -- You received this message because you are subscribed to the Google Groups Puppet Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CAOfYMj6_psV4F9xQ5KcFrVcQsstTwsRyz4-jBu%3D15ebOjHKxVA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
[Puppet-dev] Re: Thoughts on Module Namespace Conflicts
On 2015-10-02 21:17, Trevor Vaughan wrote:
To Henrik: Yes, this make sense, it would be nice to have some easy way
to keep the purely Puppet DSL portions separate though.
Yeah, but that is the way it works - so way to much to change to make
that happen.
I guess it would be possible to have a special implementation of a
module that is a proxy for a real selected module, and that the real
module is not on the real module path - only the proxy. Still quite
complex to achieve. Since the concept of a module is not that well
encapsulated.
Another approach is to differentiate between available modules (more
like a repo) and modules being used in an environment - having a
distinct step in between that configures the modulepath for the runtime.
Still the same issue with where the description of which modules to use
is kept - probably ends up with the same amount of configuration as when
just using directory environments.
You could write something yourself that does that for you - i.e.
something that creates the environments you need/want.
- henrik
Walter: This would be nice, but variables are the...uh...variable here.
Many people only expose the variables that they need while others expose
every variable they can find. It would be an interesting experiment to
see if it would work though.
Trevor
On Tue, Feb 10, 2015 at 1:52 PM, Walter Heck walterh...@olindata.com
mailto:walterh...@olindata.com wrote:
I'm personally more of a fan of what some programming languages call
interfaces; a sort of contract if you will that a module implements.
So for instance olindata::galera could look for any class that
implements the IMysql interface. That way, I don't really need to
worry if the end-user is using puppetlabs::mysql or
example42::mysql, as long as they implement the IMysql interface I
can guarantee it does what I need.
This would have pretty heavy impact on how you write code right now,
but it would be a rather elegant solution imho.
interface IMysql::server identified by
'9ff3d80b-b02d-4994-b4da-e1ff205304ea' {
Service['mysql']
Package['mysql-server']
File['apache2.conf']
}
class puppetlabs::mysql implements
IMysql['9ff3d80b-b02d-4994-b4da-e1ff205304ea'] {
[.. SNIP..]
# file, service, package here like normal
}
class example42::mysql implements
IMysql['9ff3d80b-b02d-4994-b4da-e1ff205304ea'] {
[.. SNIP..]
# file, service, package here like normal
}
class olindata::galera {
package { 'galera':
require = Package['IMysql::server::mysql-server']
}
}
Or something along those lines. Question remains where the
IMysql::server interface then needs to be declared, and how we would
manage the GUID registration of interfaces.
Comments, thoughts?
Walter
On Tuesday, February 10, 2015 at 5:52:37 PM UTC+1, henrik lindberg
wrote:
On 2015-10-02 1:23, Trevor Vaughan wrote:
I was talking with a few folks today about potential
resolutions to
module namespace issues.
== Fundamental Issue ==
puppetlabs_apache -- Installs To -- apache
example42_apache -- Installs To -- apache
theforeman_apache -- Installs To -- apache
You get my point...
== Current Solutions ==
Right now, there are two ways to solve this problem if you
want some of
your nodes to use puppetlabs_apache and others to use
theforeman_apache.
1) Modify the module and force the namespace:
- puppetlabs_apache -- Installs To -- puppetlabs_apache
- theforeman_apache -- Installs To -- theforeman_apache
* Isssue: You can't just use outside code at will.. You
have to modify
*everything* that uses the above code.
2) You have to create a separate environment for each host group
(potentially each host) that you want to have use the
differing code.
* Issue: This is a LOT of overhead for a couple of
modules and may
have other ramifications in terms of performance as you get
up in number.
So, would it be possible to allow multiple versions of a
module to exist
in the same namespace but only use one during a given run?
Basically no, this is not possible without adding some kind of
mechanism
to filter out modules on the modulepath. This will need to be
done per
environment anyway. Remember that modules can contribute all
sorts of
extensions to puppet (faces, indirections, facts, functions,
types, and
with future parser also bindings). For performance reasons (and
bootstrapping, etc.) these are scanned
Re: [Puppet-dev] Re: Thoughts on Module Namespace Conflicts
Hi Henrik, On Tue, Feb 10, 2015 at 9:30 PM, Henrik Lindberg henrik.lindb...@cloudsmith.com wrote: On 2015-10-02 19:52, Walter Heck wrote: I'm personally more of a fan of what some programming languages call interfaces; a sort of contract if you will that a module implements. Yes that is good - does not solve the issue of modules having the same name though - either the module name must change, or modules must be installed with both author and name and then referenced the same way. But with this you wouldn't need to have more then one module with the same name? All you'd need is a single module that satisfies all the interfaces the rest of your code asks for. I would think that if this was implemented, the community would gravitate towards a single IMysql interface which can then be implemented by different mysql modules in different ways. cheers, -- Best regards, Walter Heck CEO / Founder OlinData https://t.yesware.com/tl/2627942011ccc3835f76c6e9ba4c0af91f3d3722/53f65fbbd041f0e792b2eaca63996aba/53e6d075ec31fd4e14e81c226c7f32cd?ytl=http%3A%2F%2Folindata.com%2F%3Fsrc%3Dwh_gapp - Open Source Training Consulting Check out our upcoming trainings https://t.yesware.com/tl/2627942011ccc3835f76c6e9ba4c0af91f3d3722/53f65fbbd041f0e792b2eaca63996aba/50b3fd816e9110607700dc6c8a41a2cb?ytl=http%3A%2F%2Folindata.com%2Ftraining%2Fupcoming -- You received this message because you are subscribed to the Google Groups Puppet Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CAOfYMj4uk2OUtQ5tbG6waX0K4FQRfUST8Q2hf-sfp5Pz%3DsADJw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: [Puppet-dev] Re: Thoughts on Module Namespace Conflicts
To Henrik: Yes, this make sense, it would be nice to have some easy way to
keep the purely Puppet DSL portions separate though.
Walter: This would be nice, but variables are the...uh...variable here.
Many people only expose the variables that they need while others expose
every variable they can find. It would be an interesting experiment to see
if it would work though.
Trevor
On Tue, Feb 10, 2015 at 1:52 PM, Walter Heck walterh...@olindata.com
wrote:
I'm personally more of a fan of what some programming languages call
interfaces; a sort of contract if you will that a module implements.
So for instance olindata::galera could look for any class that implements
the IMysql interface. That way, I don't really need to worry if the
end-user is using puppetlabs::mysql or example42::mysql, as long as they
implement the IMysql interface I can guarantee it does what I need.
This would have pretty heavy impact on how you write code right now, but
it would be a rather elegant solution imho.
interface IMysql::server identified by
'9ff3d80b-b02d-4994-b4da-e1ff205304ea' {
Service['mysql']
Package['mysql-server']
File['apache2.conf']
}
class puppetlabs::mysql implements
IMysql['9ff3d80b-b02d-4994-b4da-e1ff205304ea'] {
[.. SNIP..]
# file, service, package here like normal
}
class example42::mysql implements
IMysql['9ff3d80b-b02d-4994-b4da-e1ff205304ea'] {
[.. SNIP..]
# file, service, package here like normal
}
class olindata::galera {
package { 'galera':
require = Package['IMysql::server::mysql-server']
}
}
Or something along those lines. Question remains where the IMysql::server
interface then needs to be declared, and how we would manage the GUID
registration of interfaces.
Comments, thoughts?
Walter
On Tuesday, February 10, 2015 at 5:52:37 PM UTC+1, henrik lindberg wrote:
On 2015-10-02 1:23, Trevor Vaughan wrote:
I was talking with a few folks today about potential resolutions to
module namespace issues.
== Fundamental Issue ==
puppetlabs_apache -- Installs To -- apache
example42_apache -- Installs To -- apache
theforeman_apache -- Installs To -- apache
You get my point...
== Current Solutions ==
Right now, there are two ways to solve this problem if you want some of
your nodes to use puppetlabs_apache and others to use
theforeman_apache.
1) Modify the module and force the namespace:
- puppetlabs_apache -- Installs To -- puppetlabs_apache
- theforeman_apache -- Installs To -- theforeman_apache
* Isssue: You can't just use outside code at will. You have to
modify
*everything* that uses the above code.
2) You have to create a separate environment for each host group
(potentially each host) that you want to have use the differing code.
* Issue: This is a LOT of overhead for a couple of modules and may
have other ramifications in terms of performance as you get up in
number.
So, would it be possible to allow multiple versions of a module to
exist
in the same namespace but only use one during a given run?
Basically no, this is not possible without adding some kind of mechanism
to filter out modules on the modulepath. This will need to be done per
environment anyway. Remember that modules can contribute all sorts of
extensions to puppet (faces, indirections, facts, functions, types, and
with future parser also bindings). For performance reasons (and
bootstrapping, etc.) these are scanned when the environment loads (some
scans are lazy, but may take place before manifests really starts to be
evaluated).
- henrik
== The First Proposal ==
Inspired by RVM Gemsets, how about allowing modules to declare which
version of a given module they will use?
/etc/puppet/modules/apache/puppetlabs/{files,modules,manifests}
/etc/puppet/modules/apache/theforeman/{files,modules,manifests}
/etc/puppet/modules/apache/author/{files,modules,manifests}
Then, use could be dictated by something like: include
apache@puppetlabs.
Unfortunately, this really comes down to the parser being able to
understand the following:
include apache = See if there are any @'s and use that one
include apache@puppetlabs = include the Puppetlabs apache
include apache@puppetlabs include apache@theforeman = Fail,
conflict
== Alternative ==
One possible alternative is to just use the metadata.json file to
dictate which module will be used when loading other modules.
Again, if there is a conflict, that is a failure but *only* if the code
attempts to use both at the same time.
The benefit here is that it should make things very unambiguous while
the drawback (if it really is) is that you *must* put a metadata.json
file in every module that you create.
This is just a set of thoughts that I hope get us moving in a direction
where this type of thing is possible and I look forward to hearing what
people think (good, bad, or ugly)!
Thanks,
[Puppet-dev] Re: Thoughts on Module Namespace Conflicts
On 2015-10-02 19:52, Walter Heck wrote:
I'm personally more of a fan of what some programming languages call
interfaces; a sort of contract if you will that a module implements.
Yes that is good - does not solve the issue of modules having the same
name though - either the module name must change, or modules must be
installed with both author and name and then referenced the same way.
- henrik
So for instance olindata::galera could look for any class that
implements the IMysql interface. That way, I don't really need to worry
if the end-user is using puppetlabs::mysql or example42::mysql, as long
as they implement the IMysql interface I can guarantee it does what I need.
This would have pretty heavy impact on how you write code right now, but
it would be a rather elegant solution imho.
interface IMysql::server identified by
'9ff3d80b-b02d-4994-b4da-e1ff205304ea' {
Service['mysql']
Package['mysql-server']
File['apache2.conf']
}
class puppetlabs::mysql implements
IMysql['9ff3d80b-b02d-4994-b4da-e1ff205304ea'] {
[.. SNIP..]
# file, service, package here like normal
}
class example42::mysql implements
IMysql['9ff3d80b-b02d-4994-b4da-e1ff205304ea'] {
[.. SNIP..]
# file, service, package here like normal
}
class olindata::galera {
package { 'galera':
require = Package['IMysql::server::mysql-server']
}
}
Or something along those lines. Question remains where the
IMysql::server interface then needs to be declared, and how we would
manage the GUID registration of interfaces.
Comments, thoughts?
Walter
On Tuesday, February 10, 2015 at 5:52:37 PM UTC+1, henrik lindberg wrote:
On 2015-10-02 1:23, Trevor Vaughan wrote:
I was talking with a few folks today about potential resolutions to
module namespace issues.
== Fundamental Issue ==
puppetlabs_apache -- Installs To -- apache
example42_apache -- Installs To -- apache
theforeman_apache -- Installs To -- apache
You get my point...
== Current Solutions ==
Right now, there are two ways to solve this problem if you want
some of
your nodes to use puppetlabs_apache and others to use
theforeman_apache.
1) Modify the module and force the namespace:
- puppetlabs_apache -- Installs To -- puppetlabs_apache
- theforeman_apache -- Installs To -- theforeman_apache
* Isssue: You can't just use outside code at will. You have to
modify
*everything* that uses the above code.
2) You have to create a separate environment for each host group
(potentially each host) that you want to have use the differing
code.
* Issue: This is a LOT of overhead for a couple of modules
and may
have other ramifications in terms of performance as you get up in
number.
So, would it be possible to allow multiple versions of a module
to exist
in the same namespace but only use one during a given run?
Basically no, this is not possible without adding some kind of
mechanism
to filter out modules on the modulepath. This will need to be done per
environment anyway. Remember that modules can contribute all sorts of
extensions to puppet (faces, indirections, facts, functions, types, and
with future parser also bindings). For performance reasons (and
bootstrapping, etc.) these are scanned when the environment loads (some
scans are lazy, but may take place before manifests really starts to be
evaluated).
- henrik
== The First Proposal ==
Inspired by RVM Gemsets, how about allowing modules to declare which
version of a given module they will use?
/etc/puppet/modules/apache/puppetlabs/{files,modules,manifests}
/etc/puppet/modules/apache/theforeman/{files,modules,manifests}
/etc/puppet/modules/apache/author/{files,modules,manifests}
Then, use could be dictated by something like: include
apache@puppetlabs.
Unfortunately, this really comes down to the parser being able to
understand the following:
include apache = See if there are any @'s and use that one
include apache@puppetlabs = include the Puppetlabs apache
include apache@puppetlabs include apache@theforeman = Fail,
conflict
== Alternative ==
One possible alternative is to just use the metadata.json file to
dictate which module will be used when loading other modules.
Again, if there is a conflict, that is a failure but *only* if
the code
attempts to use both at the same time.
The benefit here is that it should make things very unambiguous
while
the drawback (if it really is) is that you *must* put a
metadata.json
file in every module that you create.
This is just a set of thoughts that I hope get us moving in a
direction
where this
