Re: [Puppet Users] Ruby script to download files without 'puppet agent'
On Mon, Jun 27, 2011 at 7:27 AM, zu...@puzzle.ch wrote:
We have often the Problem that some files need to be checked for
updates
faster than the cycle of the puppet agent. I try to solve this with a
script which tries to download the files directly from the fileserver
of
the puppetmaster. So far i couldn't get it to work.
Have you considered using tags instead to filter for a subset of your
resources?
I need to check 1 file and 1 directory every 5min from 600 puppet nodes.
As far as i know the puppetmaster still needs to compile the whole
catalog. I did not test this, but i think this would not scale well.
The solution to this was easier than expected.
I simply write the folowing file and execute it with puppet apply
-
$source = 'puppet.example.com'
file{'/etc/sudoers':
source = [ puppet://$source/files/sudo/sudoers/${fqdn}/sudoers,
puppet://$source/files/sudo/sudoers/sudoers,
puppet://$source/sudo/sudoers/${operatingsystem}/sudoers,
puppet://$source/sudo/sudoers/sudoers ],
owner = root, group = 0, mode = 0440;
}
-
It then just deploys this one file without the need of compiling a
catalog. I can call this from cron then as often as i wish.
I saw that some use the shebang #!/usr/bin/puppet apply to start such
manifests directly as a script. This does not seam to work for me. All i
get is:
./puppet_sync_sudo2.pp: line 3: =: command not found
./puppet_sync_sudo2.pp: line 5: file{/tmp/sudoers:: No such file or directory
./puppet_sync_sudo2.pp: line 6: =: No such file or directory
./puppet_sync_sudo2.pp: line 7: puppet:///files/sudo/sudoers/sudoers,: No
such file or directory
./puppet_sync_sudo2.pp: line 8: puppet:///sudo/sudoers//sudoers,: No such
file or directory
./puppet_sync_sudo2.pp: line 9: puppet:///sudo/sudoers/sudoers: No such
file or directory
./puppet_sync_sudo2.pp: line 10: owner: command not found
./puppet_sync_sudo2.pp: line 11: syntax error near unexpected token `}'
./puppet_sync_sudo2.pp: line 11: `}'
Any ideas?
Greetings
Andy
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Ruby script to download files without 'puppet agent'
On 07/18/2011 11:27 AM, zu...@puzzle.ch wrote: I saw that some use the shebang #!/usr/bin/puppet apply to start such manifests directly as a script. This does not seam to work for me. All i get is: If your script name is foo, and you have #!/usr/bin/puppet apply as the first line, then you can run the script with chmod +x foo ./foo what will not work is: sh foo it fails because it is not a shell script. -- vagn -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Ruby script to download files without 'puppet agent'
# make ssl request connection = Net::HTTP.new(url.host, url.port) connection.use_ssl = true connection.cert = OpenSSL::X509::Certificate.new(File.read(certpath)) connection.key = OpenSSL::PKey::RSA.new(File.read(pkey_path)) you are missing the CA file... something like connection.ca_file = Puppet[:localcacert] Tested that, but it seams to make no difference. From what i unerstand the ca is not really needed for this on the client. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Ruby script to download files without 'puppet agent'
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 [root@foo me]# ./puppet-wget file_content/sudo/sudoers https://foo.bar:8140/file_content/sudo/sudoers warning: peer certificate won't be verified in this SSL session /usr/lib/ruby/1.8/net/http.rb:2099:in `error!': 403 Forbidden request: foo.bar(10.2.3.4) access to /sudo/sudoers [find] authenticated at line 93 (Net::HTTPServerException) from ./puppet-wget:56 from ./puppet-wget:43:in `each' from ./puppet-wget:43 Using your script works to get a file from a module files, like: ./puppet_wget /development/file_content/modules/site-bind/etc/rz.foo.ch/named.conf However your error looks like the client is not authorized to find that content. Maybe some tweaks in the auth.conf are missing to fetch files from the fileserver instead from modules? ~pete -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk4LCqYACgkQbwltcAfKi38O9wCgnUKnyugLiukznBcjjLQ2wXbd sK0AnA8UAYkrIiFPAkIbCELvFLu7F6q3 =iiOQ -END PGP SIGNATURE- -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Ruby script to download files without 'puppet agent'
We have often the Problem that some files need to be checked for updates faster than the cycle of the puppet agent. I try to solve this with a script which tries to download the files directly from the fileserver of the puppetmaster. So far i couldn't get it to work. I don't know if i got the URL right, i did not find any examples on the REST API documentation for the fileserver. for example if the file is /etc/puppet/manifests/files/sudo/sudoers fileserver.conf: [files] path /etc/puppet/manifests/files allow 127.0.0.1 allow *.bar to what path would this translate? https://foo.bar:8140/file_content/sudo/sudoers https://bli.bla:8140/file_content/files/sudo/sudoers Greetings Andy -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Ruby script to download files without 'puppet agent'
On Mon, Jun 27, 2011 at 7:27 AM, zu...@puzzle.ch wrote: We have often the Problem that some files need to be checked for updates faster than the cycle of the puppet agent. I try to solve this with a script which tries to download the files directly from the fileserver of the puppetmaster. So far i couldn't get it to work. Have you considered using tags instead to filter for a subset of your resources? I don't know if i got the URL right, i did not find any examples on the REST API documentation for the fileserver. This should cover it. http://docs.puppetlabs.com/guides/rest_api.html#file-server -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Ruby script to download files without 'puppet agent'
On Mon, Jun 27, 2011 at 7:27 AM, zu...@puzzle.ch wrote: We have often the Problem that some files need to be checked for updates faster than the cycle of the puppet agent. I try to solve this with a script which tries to download the files directly from the fileserver of the puppetmaster. So far i couldn't get it to work. Have you considered using tags instead to filter for a subset of your resources? I need to check 1 file and 1 directory every 5min from 600 puppet nodes. As far as i know the puppetmaster still needs to compile the whole catalog. I did not test this, but i think this would not scale well. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Ruby script to download files without 'puppet agent'
On Mon, Jun 27, 2011 at 7:41 AM, zu...@puzzle.ch wrote: On Mon, Jun 27, 2011 at 7:27 AM, zu...@puzzle.ch wrote: We have often the Problem that some files need to be checked for updates faster than the cycle of the puppet agent. I try to solve this with a script which tries to download the files directly from the fileserver of the puppetmaster. So far i couldn't get it to work. Have you considered using tags instead to filter for a subset of your resources? I need to check 1 file and 1 directory every 5min from 600 puppet nodes. As far as i know the puppetmaster still needs to compile the whole catalog. I did not test this, but i think this would not scale well. Possibly. You could also do something like this. Put the thing you need to check frequently into a module, we'll call it frequent for now, and structure your setup with environments like the following: # normal environment [normal] modulepath=/var/lib/puppet/env/normal/modules:/var/lib/puppet/env/frequent/modules [frequent] modulepath=/var/lib/puppet/env/frequent/modules Then you could choose to run every 5 minutes against the frequent environment, and you'd have a very minimal catalog for the rapid runs, and a more comprehensive one for your normal runs. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Ruby script to download files without 'puppet agent'
On Mon, Jun 27, 2011 at 7:27 AM, zu...@puzzle.ch wrote:
I don't know if i got the URL right, i did not find any examples on the
REST API documentation for the fileserver.
This should cover it.
http://docs.puppetlabs.com/guides/rest_api.html#file-server
This isn't still clear to me. Would appreciate if someone could give a
concrete example how a path in a module or in the filepath would tanslate.
Here is what i try:
fileserver.conf
[files]
path /etc/puppet/manifests/files
allow 127.0.0.1
allow *.smobi.mobicorp.test
The file is located at /etc/puppet/manifests/files/sudo/sudoers
[root@foo me]# ./puppet-wget file_content/sudo/sudoers
https://foo.bar:8140/file_content/sudo/sudoers
warning: peer certificate won't be verified in this SSL session
/usr/lib/ruby/1.8/net/http.rb:2099:in `error!': 403 Forbidden request:
foo.bar(10.2.3.4) access to /sudo/sudoers [find] authenticated at line 93
(Net::HTTPServerException)
from ./puppet-wget:56
from ./puppet-wget:43:in `each'
from ./puppet-wget:43
AFAIK if something is wrong with the certs the server would answer with a
401 and not 403. It really looks to me as if i get that path wrong.
--
#!/bin/env ruby
#
# Puppet Wget
#
# Downloads files from the puppetmaster without the puppet agent
require 'optparse'
require 'puppet/rails'
require 'uri'
require 'net/https'
# Parse all Options
options = {}
OptionParser.new do |opts|
opts.banner = Usage: puppet-wget [options] module/path/to/file
opts.on(-r, --recursive, Download directory recursively) do |r|
options[:recursive] = r
end
end.parse!
# Get some settings for the puppet config
Puppet[:config] = /etc/puppet/puppet.conf
Puppet.parse_config
puppet_conf = Puppet.settings.instance_variable_get(:@values)[:main]
# default values
puppet_conf[:server] ||= 'puppet'
puppet_conf[:masterport] ||= '8140'
puppet_conf[:ssldir] ||= '/etc/puppet/ssl'
server= puppet_conf[:server]
port = puppet_conf[:masterport]
certpath = puppet_conf[:ssldir] + '/certs/'+ ENV['HOSTNAME'] +
'.pem'
pkey_path = puppet_conf[:ssldir] + '/private_keys/' + ENV['HOSTNAME'] +
'.pem'
ARGV.each do |filepath|
url = URI.parse(https://#{server}:#{port}/#{filepath};)
req = Net::HTTP::Get.new(#{url.path}?#{url.query}, Accept = 's')
puts url
# make ssl request
connection = Net::HTTP.new(url.host, url.port)
connection.use_ssl = true
connection.cert = OpenSSL::X509::Certificate.new(File.read(certpath))
connection.key = OpenSSL::PKey::RSA.new(File.read(pkey_path))
res = connection.start { |http| http.request(req) }
res.error! unless res.code_type == Net::HTTPOK
puts res.body
end
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Ruby script to download files without 'puppet agent'
On Mon, Jun 27, 2011 at 5:55 PM, zu...@puzzle.ch wrote:
On Mon, Jun 27, 2011 at 7:27 AM, zu...@puzzle.ch wrote:
I don't know if i got the URL right, i did not find any examples on the
REST API documentation for the fileserver.
This should cover it.
http://docs.puppetlabs.com/guides/rest_api.html#file-server
This isn't still clear to me. Would appreciate if someone could give a
concrete example how a path in a module or in the filepath would tanslate.
Here is what i try:
fileserver.conf
[files]
path /etc/puppet/manifests/files
allow 127.0.0.1
allow *.smobi.mobicorp.test
The file is located at /etc/puppet/manifests/files/sudo/sudoers
[root@foo me]# ./puppet-wget file_content/sudo/sudoers
https://foo.bar:8140/file_content/sudo/sudoers
warning: peer certificate won't be verified in this SSL session
/usr/lib/ruby/1.8/net/http.rb:2099:in `error!': 403 Forbidden request:
foo.bar(10.2.3.4) access to /sudo/sudoers [find] authenticated at line 93
(Net::HTTPServerException)
from ./puppet-wget:56
from ./puppet-wget:43:in `each'
from ./puppet-wget:43
AFAIK if something is wrong with the certs the server would answer with a
401 and not 403. It really looks to me as if i get that path wrong.
--
#!/bin/env ruby
#
# Puppet Wget
#
# Downloads files from the puppetmaster without the puppet agent
require 'optparse'
require 'puppet/rails'
require 'uri'
require 'net/https'
# Parse all Options
options = {}
OptionParser.new do |opts|
opts.banner = Usage: puppet-wget [options] module/path/to/file
opts.on(-r, --recursive, Download directory recursively) do |r|
options[:recursive] = r
end
end.parse!
# Get some settings for the puppet config
Puppet[:config] = /etc/puppet/puppet.conf
Puppet.parse_config
puppet_conf = Puppet.settings.instance_variable_get(:@values)[:main]
# default values
puppet_conf[:server] ||= 'puppet'
puppet_conf[:masterport] ||= '8140'
puppet_conf[:ssldir] ||= '/etc/puppet/ssl'
server = puppet_conf[:server]
port = puppet_conf[:masterport]
certpath = puppet_conf[:ssldir] + '/certs/' + ENV['HOSTNAME'] +
'.pem'
pkey_path = puppet_conf[:ssldir] + '/private_keys/' + ENV['HOSTNAME'] +
'.pem'
ARGV.each do |filepath|
url = URI.parse(https://#{server}:#{port}/#{filepath};)
req = Net::HTTP::Get.new(#{url.path}?#{url.query}, Accept = 's')
puts url
# make ssl request
connection = Net::HTTP.new(url.host, url.port)
connection.use_ssl = true
connection.cert = OpenSSL::X509::Certificate.new(File.read(certpath))
connection.key = OpenSSL::PKey::RSA.new(File.read(pkey_path))
you are missing the CA file... something like
connection.ca_file = Puppet[:localcacert]
res = connection.start { |http| http.request(req) }
res.error! unless res.code_type == Net::HTTPOK
puts res.body
end
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
