Jira (PDB-4579) Enable tcpKeepAlive on the postgres driver
Title: Message Title
Taylan Develioglu updated an issue
PuppetDB / PDB-4579
Enable tcpKeepAlive on the postgres driver
Change By:
Taylan Develioglu
Bringing down the network interface link on our the PostgreSQL server causes the connection pool to hold on to connections that were already closed on the database server for (what seems t oe to be) an infinite time.The network stack on the client is never notified these connections have been closed on the peer and PuppetDB's connection pool still believes they are active.This caused us to run out of available connections in the connection pool until restarting PuppetDB. The PDBReadPool_pool_ActiveConnections metric also reports a value of 25 (maximum-pool-size). Can the tcpKeepAlive option of the PostgreSQL JDBC driver be enabled to prevent this class of issue from happening ? h5. Network link going down on the PostgreSQL server {code:java}[di nov 5 12:49:51 2019] bnx2x :37:00.0 eno1: NIC Link is Down[di nov 5 12:55:03 2019] bnx2x :37:00.0 eno1: NIC Link is Up, 1 Mbps full duplex, Flow control: none[di nov 5 12:55:09 2019] bnx2x :37:00.0 eno1: NIC Link is Down[di nov 5 12:55:10 2019] bnx2x :37:00.0 eno1: NIC Link is Up, 1 Mbps full duplex, Flow control: none[di nov 5 12:55:11 2019] bnx2x :37:00.0 eno1: NIC Link is Down[di nov 5 12:55:13 2019] bnx2x :37:00.0 eno1: NIC Link is Up, 1 Mbps full duplex, Flow control: none{code} h5. Connections still in ESTABLISHED state on the client side {code:java}[root@puppetdb ~]# netstat -ntp|grep 10.197.29.74:5432tcp6 0 0 10.198.174.11:39186 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:59996 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:50380 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:60952 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:33536 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:60902 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:35564 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:57950 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:45416 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:33644 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:39678 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:43846 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:55738 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:58098 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:34214 10.197.29.74:5432
Jira (PDB-4579) Enable tcpKeepAlive on the postgres driver
Title: Message Title
Taylan Develioglu updated an issue
PuppetDB / PDB-4579
Enable tcpKeepAlive on the postgres driver
Change By:
Taylan Develioglu
Bringing down the network interface on our PostgreSQL server causes the connection pool to hold on to connections that were already closed on the database server for (what seems t oe be) an infinite time.The network stack on the client is never notified these connections have been closed on the peer and PuppetDB's connection pool still believes they are active.This caused us to run out of available connections in the connection pool until restarting PuppetDB. The PDBReadPool_pool_ActiveConnections metric also reports a value of 25 (maximum-pool-size). Can the tcpKeepAlive option of the PostgreSQL JDBC driver be enabled to prevent this class of issue from happening ? h5. Network link going down on the PostgreSQL server {code:java} [di nov 5 12:49:51 2019] bnx2x :37:00.0 eno1: NIC Link is Down[di nov 5 12:55:03 2019] bnx2x :37:00.0 eno1: NIC Link is Up, 1 Mbps full duplex, Flow control: none[di nov 5 12:55:09 2019] bnx2x :37:00.0 eno1: NIC Link is Down[di nov 5 12:55:10 2019] bnx2x :37:00.0 eno1: NIC Link is Up, 1 Mbps full duplex, Flow control: none[di nov 5 12:55:11 2019] bnx2x :37:00.0 eno1: NIC Link is Down[di nov 5 12:55:13 2019] bnx2x :37:00.0 eno1: NIC Link is Up, 1 Mbps full duplex, Flow control: none{code} h5. Connections still in ESTABLISHED state on the client side {code:java} [root@puppetdb ~]# netstat -ntp|grep 10.197.29.74:5432tcp6 0 0 10.198.174.11:39186 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:59996 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:50380 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:60952 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:33536 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:60902 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:35564 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:57950 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:45416 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:33644 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:39678 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:43846 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:55738 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:58098 10.197.29.74:5432 ESTABLISHED 47079/java tcp6 0 0 10.198.174.11:34214 10.197.29.74:5432 ESTABLISHED
Jira (PUP-7738) "bundle exec puppet module install" fails on Puppet 5.0.0
Title: Message Title Taylan Develioglu commented on PUP-7738 Re: "bundle exec puppet module install" fails on Puppet 5.0.0 Josh Cooper Can confirm this fixes the issue. Add Comment This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PUP-3732) type validation should not run for `puppet resource`
Title: Message Title Taylan Develioglu commented on PUP-3732 Re: type validation should not run for `puppet resource` Just wanted to chime that you can guard against this behavior by checking if the resource is managed. e.g.: validate do if managed? unless self[:value] raise ArgumentError, 'Active_value must be set' end end end Add Comment
Jira (HI-545) Environment data lookup stops processing hierarchy if sub_lookup for interpolation token can't find segment
Title: Message Title
Taylan Develioglu updated an issue
Hiera / HI-545
Environment data lookup stops processing hierarchy if sub_lookup for interpolation token can't find segment
Change By:
Taylan Develioglu
If you have a hierarchy level that needs to interpolate a token that is a nested hash key but that key doesn't exist, the lookup silently ignores remaining hierarchy levels.In the following (environment) hiera.yaml there is no 'cluster' key in the 'serverdb' fact. The result is that lookup will ignore all levels after "ServerDB cluster".This is a severe issue, because a missing key could potentially break the entire infrastructure by reverting to module defaults.The relevant bit that I could find is in https://github.com/puppetlabs/puppet/blob/master/lib/puppet/pops/lookup/sub_lookup.rb#L73-L75, after a throw while looking for the 'cluster' segment it immediately proceeds to (in our case) lookup module_data.This is repeated for each class parameter and we end up with nothing but default values.{code:yaml}---version: 4datadir: datahierarchy: - name: "Trusted certname"backend: yamlpath: "certname/%{::trusted.certname}" - name: "FQDN"backend: yamlpath: "fqdn/%{::facts.fqdn}" - name: "ServerDB cluster"backend: yamlpath: "cluster/%{::facts.serverdb.cluster}" - name: "ServerDB role"backend: yamlpaths:- "role/%{::facts.serverdb.role}/os/%{facts.os.name}/%{facts.os.release.major}"- "role/%{::facts.serverdb.role}/domain/%{::facts.domain}"- "role/%{::facts.serverdb.role}" - name: "ServerDB environment"backend: yamlpath: "environment/%{::facts.serverdb.environment}" - name: "Network"backend: yamlpath: "network/%{::facts.network_16}/%{::facts.network_24}"path: "network/%{::facts.network_16}" - name: "Domain"backend: yamlpath: "domain/%{::facts.domain}" - name: "common"backend: yamlpath: "common"{code}To be safe, the run should halt with an error , or at the very least proceed to process the remaining levels .
Add Comment
Jira (HI-545) Environment data lookup stops processing hierarchy if sub_lookup for interpolation token can't find segment
Title: Message Title Taylan Develioglu created an issue Hiera / HI-545 Environment data lookup stops processing hierarchy if sub_lookup for interpolation token can't find segment Issue Type: Bug Assignee: Unassigned Created: 2016/11/22 5:55 AM Priority: Major Reporter: Taylan Develioglu If you have a hierarchy level that needs to interpolate a token that is a nested hash key but that key doesn't exist, the lookup silently ignores remaining hierarchy levels. In the following (environment) hiera.yaml there is no 'cluster' key in the 'serverdb' fact. The result is that lookup will ignore all levels after "ServerDB cluster". This is a severe issue, because a missing key could potentially break the entire infrastructure by reverting to module defaults. The relevant bit that I could find is in https://github.com/puppetlabs/puppet/blob/master/lib/puppet/pops/lookup/sub_lookup.rb#L73-L75, after a throw while looking for the 'cluster' segment it immediately proceeds to (in our case) lookup module_data. This is repeated for each class parameter and we end up with nothing but default values. --- version: 4
Jira (PUP-5062) static compiler fails to remove existing resources from children.
Title: Message Title
Taylan Develioglu created an issue
Puppet / PUP-5062
static compiler fails to remove existing resources from children.
Issue Type:
Bug
Assignee:
Unassigned
Created:
2015/08/21 4:22 AM
Priority:
Normal
Reporter:
Taylan Develioglu
The static compiler fails to remove child resources that already exist in the catalog.
file { '/tmp/statically_compiled':
ensure = directory,
owner = 'root',
group = 'root',
Jira (PUP-5055) parent attributes are set from metadata too early in static_compiler
Title: Message Title Taylan Develioglu created an issue Puppet / PUP-5055 parent attributes are set from metadata too early in static_compiler Issue Type: Bug Assignee: Unassigned Created: 2015/08/20 6:40 AM Priority: Normal Reporter: Taylan Develioglu in get_child_resources() we are assigning mode, owner and group attributes to the parent too early. ... total.each do |meta| # This is the top-level parent directory if meta.relative_path == .
Jira (PUP-5035) undefined method `keys' for nil:NilClass in static_compiler
Title: Message Title
Taylan Develioglu created an issue
Puppet / PUP-5035
undefined method `keys' for nil:NilClass in static_compiler
Issue Type:
Bug
Assignee:
Unassigned
Created:
2015/08/13 12:55 AM
Priority:
Normal
Reporter:
Taylan Develioglu
get_child_resources() can return nil if the source attribute of the file resource points to a file instead of a directory.
This results in a raised NoMethodError with the static compiler set as the catalog terminus and a file resource declared that has the recurse parameter set to true, e.g.:
file { '/etc/puppet.conf':
recurse = true,
source = 'puppet:///modules/puppet/puppet.conf',
}
Jira (PUP-5014) Base class for JSON indirection terminus calls log_exception() wrong.
Title: Message Title
Taylan Develioglu created an issue
Puppet / PUP-5014
Base class for JSON indirection terminus calls log_exception() wrong.
Issue Type:
Bug
Affects Versions:
PUP 3.8.1
Assignee:
Kylo Ginsberg
Components:
Client
Created:
2015/08/10 5:02 AM
Priority:
Normal
Reporter:
Taylan Develioglu
In Puppet::Indirector::JSON, Puppet.log_exception() is called without the required exception as the first argument value, instead the message is passed.
Puppet.log_exception Could not save # {self.name} #{request.key}: #{detail} While it should really be: Puppet.log_exception(detail, Could not save #{self.name}
# {request.key}
: # {detail}
)
This breaks further along in format_exception() as it attempts to get the exception message attribute but finds a string:
Error: Could not retrieve catalog from remote server: undefined method `message' for #String:0x0003efda28 /opt/ruby2.0/lib/vendor_ruby/2.0.0/puppet/util/logging.rb:70:in `format_exception' /opt/ruby2.0/lib/vendor_ruby/2.0.0/puppet/util/logging.rb:45:in `log_exception' /var/lib/puppet/lib/puppet/indirector/catalog/json_timestamped.rb:18:in `rescue in save'
Jira (PUP-5014) Base class for JSON indirection terminus calls log_exception() wrong.
Title: Message Title
Taylan Develioglu updated an issue
Puppet / PUP-5014
Base class for JSON indirection terminus calls log_exception() wrong.
Change By:
Taylan Develioglu
InPuppet::Indirector::JSON,Puppet.log_exception()iscalledwithouttherequiredexceptionasthefirstargumentvalue,insteadthemessageispassed. {code} Puppet.log_exceptionCouldnotsave#{self.name}#{request.key}:#{detail} {code} Whileitshouldreallybe: {code} Puppet.log_exception(detail,Couldnotsave#{self.name}#{request.key}:#{detail}) {code} Thisbreaksfurtheralonginformat_exception()asitattemptstogettheexceptionmessageattributebutfindsastring:Error:Couldnotretrievecatalogfromremoteserver:undefinedmethod`message'for#String:0x0003efda28/opt/ruby2.0/lib/vendor_ruby/2.0.0/puppet/util/logging.rb:70:in`format_exception'/opt/ruby2.0/lib/vendor_ruby/2.0.0/puppet/util/logging.rb:45:in`log_exception'/var/lib/puppet/lib/puppet/indirector/catalog/json_timestamped.rb:18:in`rescueinsave'/var/lib/puppet/lib/puppet/indirector/catalog/json_timestamped.rb:13:in`save'/opt/ruby2.0/lib/vendor_ruby/2.0.0/puppet/indirector/indirection.rb:206:in`find'/opt/ruby2.0/lib/vendor_ruby/2.0.0/puppet/configurer.rb:294:in`blockinretrieve_new_catalog'/opt/ruby2.0/lib/vendor_ruby/2.0.0/puppet/util.rb:335:in`blockinthinmark'/opt/ruby2.0/lib/2.0.0/benchmark.rb:296:in`realtime'/opt/ruby2.0/lib/vendor_ruby/2.0.0/puppet/util.rb:334:in`thinmark'/opt/ruby2.0/lib/vendor_ruby/2.0.0/puppet/configurer.rb:293:in`retrieve_new_catalog'/opt/ruby2.0/lib/vendor_ruby/2.0.0/puppet/configurer.rb:61:in`retrieve_catalog'/opt/ruby2.0/lib/vendor_ruby/2.0.0/puppet/configurer.rb:106:in`prepare_and_retrieve_catalog'/opt/ruby2.0/lib/vendor_ruby/2.0.0/puppet/configurer.rb:202:in`run_internal'/opt/ruby2.0/lib/vendor_ruby/2.0.0/puppet/configurer.rb:134:in`blockinrun'...
Add Comment
This message was sent by Atlassian JIRA (v6.4.5#64020-sha1:78acd6c)
Jira (PUP-4919) static compiler always sets ensure parameter to 'file' if a source is declared
Title: Message Title
Taylan Develioglu updated an issue
Puppet / PUP-4919
static compiler always sets ensure parameter to 'file' if a source is declared
Change By:
Taylan Develioglu
Thestatic_compilercatalogterminusalwayssetsensureto'file'onfileresourcesaslongastheresourcehasasourceparameterdeclared.Itdoesthiseveniftheensureparameterisexplicitlydeclared.Applyingacatalog,withthefollowingfileresource,compiledwithstaticcompiler,willcreatethefileanyway. {code} file{'/tmp/statically_compiled.txt':ensure=absent,mode=0755,owner='root',group='root',source='puppet:///modules/foo/statically_compiled.txt',} {code} ...Notice:/Stage[main]/Main/File[/tmp/statically_compiled.txt]/ensure:definedcontentas'{md5}8fea78c7e5ed0f5b3bd13565589a6f7c'... Thecatalogconfirmsensureisbeingsettofile:irb(main):001:0require'json'=trueirb(main):002:0JSON.load(File.read('foo.json'))['data']['resources'].find{|res|res['title']=='/tmp/statically_compiled.txt'}={type=File,title=/tmp/statically_compiled.txt,tags=[file,class],file=/etc/puppet/environments/tdevelioglu/puppet/manifests/site.pp,line=7,exported=false,parameters={ensure=file,mode=0755,owner=root,group=root,ignore=[.svn,CVS,.git],content={md5}8fea78c7e5ed0f5b3bd13565589a6f7c,checksum=md5}}Thisisinconsistentwiththebehaviorofthenormalcompiler,whereafile'sensureparameterisenforcedregardlessofthevalueofthesourceparameter.
Add Comment
This message was sent by Atlassian JIRA (v6.4.5#64020-sha1:78acd6c)
--
You received this message because you are subscribed to the Google Groups Puppet Bugs group.
To unsubscribe from this group and stop receiving emails
Jira (PUP-4919) static compiler always sets ensure parameter to 'file' if a source is declared
Title: Message Title
Taylan Develioglu created an issue
Puppet / PUP-4919
static compiler always sets ensure parameter to 'file' if a source is declared
Issue Type:
Bug
Affects Versions:
PUP 3.7.5
Assignee:
Unassigned
Created:
2015/07/23 5:19 AM
Priority:
Normal
Reporter:
Taylan Develioglu
The static_compiler catalog terminus always sets ensure to 'file' on file resources as long as the resource has a source parameter declared. It does this even if the ensure parameter is explicitly declared.
Applying a catalog, with the following file resource, compiled with static compiler, will create the file anyway.
file { '/tmp/statically_compiled.txt': ensure = absent, mode = 0755, owner = 'root', group = 'root', source = 'puppet:///modules/foo/statically_compiled.txt', }
... Notice: /Stage[main]/Main/File[/tmp/statically_compiled.txt]/ensure: defined content as ' {md5}8fea78c7e5ed0f5b3bd13565589a6f7c' ... The catalog confirms ensure is being set to file : irb(main):001:0 require 'json' = true irb(main):002:0 JSON.load(File.read('foo.json'))['data']['resources'].find { |res| res['title'] == '/tmp/statically_compiled.txt' } = {type=File, title=/tmp/statically_compiled.txt, tags=[file, class], file=/etc/puppet/environments/tdevelioglu/puppet/manifests/site.pp, line=7, exported=false, parameters={ensure=file, mode=0755, owner=root, group=root, ignore=[.svn, CVS, .git], content={md5}
8fea78c7e5ed0f5b3bd13565589a6f7c, checksum=md5}}
This is inconsistent
Jira (PDB-1815) PuppetDB DDL not supported when using PostgreSQL with BDR
Title: Message Title Taylan Develioglu created an issue PuppetDB / PDB-1815 PuppetDB DDL not supported when using PostgreSQL with BDR Issue Type: Improvement Affects Versions: PDB 2.3.6 Assignee: Unassigned Created: 2015/07/20 7:37 AM Priority: Normal Reporter: Taylan Develioglu When using PostgreSQL with bi-directional replication, some of the ddl statements PuppetDB uses to bring its schema up to date are not supported. Trying to bring the schema up to date from scratch results in the following error from PostgreSQL (bdr 0.9.1 on postgresql 9.4.2): ERROR: ALTER TABLE ... ALTER COLUMN TYPE may only affect UNLOGGED or TEMPORARY tables when BDR is active; reports is a regular table_ There are probably more, but this is the first one that we hit. According to the bdr documentation 'alter table .. alter column type' is prohibited, see http://bdr-project.org/docs/0.9/ddl-replication-statements.html Add
Jira (PUP-4781) filemode retrieved by static_compiler should be stringified
Title: Message Title Taylan Develioglu created an issue Puppet / PUP-4781 filemode retrieved by static_compiler should be stringified Issue Type: Bug Affects Versions: PUP 3.2.1 Assignee: Unassigned Created: 2015/06/24 6:49 AM Priority: Minor Reporter: Taylan Develioglu When a file resource is declared without an explicit mode, the mode retrieved from the metadata by the static_compiler catalog terminus is not stringified, resulting in file resources with integer file modes in the resultant catalog and a depreciation warning such as: Warning: Non-string values for the file mode property are deprecated. It must be a string, either a symbolic mode like 'o+w,a+r' or an octal representation like '0644' or '755'. (at /usr/lib/ruby/vendor_ruby/puppet/type/file/mode.rb:69:in `block (2 levels) in module:Puppet') Add Comment
