Jira (PUP-10777) augeas resource type set service-name bug when specifying a port with 5 digits

[Luchian Nemes (Jira)]

Title: Message Title

Luchian Nemes commented on PUP-10777   Re: augeas resource type set service-name bug when specifying a port with 5 digits Hello Henry Wang,   The issue presented in the ticket seems to be due to user error (investigation journey/details can be seen below). I will close this ticket with 'Won't fix' for now but feel free to ping/reopen if there is anything unclear. Following your exact steps for reproducing the issue, allowed me to get the same error message and see the different behaviours with 4 digit ports (used the same '7005') and 5 digit ports (used the same '17000'). Setting the comment part from reproducing steps ("set service-name[port = '7005'][protocol = 'udp']/#comment 'server2agent'") didn't seem relevant so I've simplified the manifest to: # simplified_manifest.pp augeas { 'myService': context => '/files/etc/services', changes => [ "set service-name[port = '12345'][protocol = 'tcp'] myService", ] }   Surely enough, applying this simplified manifest still threw the same error: ... Debug: Augeas[myService](provider=augeas): Opening augeas with root /, lens path , flags 32 Debug: Augeas[myService](provider=augeas): Augeas version 1.12.0 is installed Debug: Augeas[myService](provider=augeas): Will attempt to save and only run if files changed Debug: Augeas[myService](provider=augeas): sending command 'set' with params ["/files/etc/services/service-name[port = '12345'][protocol = 'tcp']", "myService"] Debug: Augeas[myService](provider=augeas): Put failed on one or more files, output from /augeas//error: Debug: Augeas[myService](provider=augeas): /augeas/files/etc/services/error = put_failed Debug: Augeas[myService](provider=augeas): /augeas/files/etc/services/error/path = /files/etc/services/files/etc/services/service-name[558] Debug: Augeas[myService](provider=augeas): /augeas/files/etc/services/error/lens = /opt/puppetlabs/puppet/share/augeas/lenses/dist/services.aug:80.15-83.33: Debug: Augeas[myService](provider=augeas): /augeas/files/etc/services/error/message = Failed to match tree under /files/etc/services/service-name[558] with pattern ({ /port/ = /[0-9]+/ } | { /start/ = /[0-9]+/ } { /end/ = /[0-9]+/ }) { /protocol/ = /[A-Za-z]+/ } ( { /alias/ = /[*+.-:A-Z_a-z-]+/ })* ({ /#comment/ = /[^\t\n\r ].*[^\t\n\r ]|[^\t\n\r ]/ } | ()) Debug: Augeas[myService](provider=augeas): Closed the augeas connection ...   While looking through the /etc/services file, I found that it already contained 5 digits long ports. Since the file was containing over 500 entries and I wanted to remove any OS specific doubts or regarding the file's format, I've simplified this also to: a 1/tcp # 1 digit b 12/tcp # 2 digits c 123/tcp # 3 digits d 1234/udp # 4 digits e 12345/udp # 5 digits   When running the same simplified manifest as above with the simplified /etc/services file: ... Debug: Augeas[myService](provider=augeas): Opening augeas with root /, lens path , flags 32 Debug: Augeas[myService](provider=augeas): Augeas version 1.12.0 is installed Debug: Augeas[myService](provider=augeas): Will attempt to save and only run if files changed Debug: Augeas[myService](provider=augeas): sending command 'set' with params ["/files/etc/services/service-name[port = '12345'][protocol = 'tcp']", "myService"] Debug: Augeas[myService](provider=augeas): Put failed on one or more files, output from /augeas//error: Debug: Augeas[myService](provider=augeas): /augeas/files/etc/services/error = put_failed Debug: Augeas[myService](provider=augeas): /augeas/files/etc/services/error/path = /files/etc/services/files/etc/services/service-name[6] Debug: Augeas[myService](provider=augeas): /augeas/files/etc/services/error/lens = /opt/puppetlabs/puppet/share/augeas/lenses/dist/services.aug:80.15-83.33: Debug: Augeas[myService](provider=augeas): /augeas/files/etc/services/error/message = Failed to match tree under /files/etc/services/service-name[6] with pattern (\ /port/ = /[0-9]+/ } | { /start/ = /[0-9]+/ } { /end/ = /[0-9]+/ }) { /protocol/ = /[A-Za-z]+/ } ( { /alias/ = /[*+.-:A-Z_a-z-]+/ })* ({ /#comment/ = /[^\t\n\r ].*[^\t\n\r ]|[^\t\n\r ]/ } | ()) Debug: Augeas[myService](provider=augeas): Closed the augeas connection ... At this time of investigation, it seemed to be the same error. Will return to this later on.   Using augtool to apply the same setting as in the simplified manifest: root@my-machine:~# /opt/puppetlabs/puppet/bin/augtool augtool> set /files/etc/services/service-name[port = '12345'][protocol = 'tcp'] myService augtool> save error: Failed to execute command saving failed (run 'errors' for details) augtool> errors Error in /etc/profile.d/puppet-agent.sh:5.58 (parse_failed) Syntax error Lens: /opt/puppetlabs/puppet/share/augeas/lenses/dist/shellvars.aug:251.12-.60: Error in /etc/services at node /files/etc/services/files/etc/services/service-name[6] (put_failed) Failed to match tree under /files/etc/services/service-name[6] with pattern ({ /port/ = /[0-9]+/ } | { /start/ = /[0-9]+/ } { /end/ = /[0-9]+/ }) { /protocol/ = /[A-Za-z]+/ } ( { /alias/ = /[*+.-:A-Z_a-z-]+/ })* ({ /#comment/ = /[^\t\n\r ].*[^\t\n\r ]|[^\t\n\r ]/ } | ()) Lens: /opt/puppetlabs/puppet/share/augeas/lenses/dist/services.aug:80.15-83.33: augtool> quit   Since puppetlabs/puppetlabs-augeas_core is just extracting data from the given manifest and translates it into augeas calls, I've continued my search upstream in the hercules-team/augeas code. From all of the runs above, we can see that both augtool and puppet apply use the same lense: from applying the simplified manifest: Debug: Augeas[myService](provider=augeas): /augeas/files/etc/services/error/lens = /opt/puppetlabs/puppet/share/augeas/lenses/dist/services.aug:80.15-83.33: from augtool run: Lens: /opt/puppetlabs/puppet/share/augeas/lenses/dist/services.aug:80.15-83.33:   In the lense file we can see that the port setting has no number of digits restriction (the num_re regular _expression_ matches 1 or any number of digits): let num_re = /[0-9]+/ ... let port = [ label "port" . store num_re ]   In the history of the file on the repository, we can see that this part of the lense didn't change since the initial commit from 12 years ago so the issue doesn't depend on the version of augeas (https://github.com/hercules-team/augeas/blame/master/lenses/services.aug#L51-L62).   Going back to changing the simplified manifest to use a 4 digits long port (tried with '1234' and '7005' as initially) started throwing the same error (with the simplified /etc/services file) as when using a 5 digits long port (with the original /etc/services file). Looking at the "Failed to match tree under /files/etc/services/service-name" from debug messages and the number of entries in the file we can see that it is always follow by that +1: /etc/services content: a 1/tcp # 1 digit b 12/tcp # 2 digits c 123/tcp # 3 digits d 1234/udp # 4 digits e 12345/udp # 5 digits from debug messages: Failed to match tree under /files/etc/services/service-name[6] # applied manifest: augeas { 'myService': context => '/files/etc/services', changes => [ "set service-name[port = '12345'][protocol = 'tcp'] myService", ] } This happens because in the applied manifest we're actually asking augeas to first search for a specific entry and then try to change it if necessary. We're searching for an entry where the port is already set to '12345' *and protocol is already set to *'tcp' and when found (not if), look at the name and change it to 'myService' if it's not already 'myService'. If we look at the /etc/services file contents we see that we have service 'e' with port '12345' but the protocol is actually 'udp' (instead of 'tcp' as written in the manifest). So our issue is actually an out-of-range search sort of error than anything else since it can't find any entry with port '12345' and protocol 'tcp'. It has nothing to do with the number of digits in the port field.   A correct/working form of the manifest would be: augeas { 'myService': context => '/files/etc/services', changes => [ "set service-name[. = 'serviceName'] 'serviceName'", "set service-name[. = 'serviceName']/port '12345'", "set service-name[. = 'serviceName']/protocol 'tcp'", "set service-name[. = 'serviceName']/#comment '5 digits'", ] } This would first add 'serviceName' at the end of the file if it doesn't already exist and then configure everything as given. The downside of this manifest (as it is here) would be the risk of duplicating the port+protocol entry in the /etc/services file but it would be a valid call for the augeas module.   Another correct/working variant of the manifest would be: augeas { 'myService': context => '/files/etc/services', changes => [ "set service-name[port = '12345'][protocol = 'tcp'] myService", ], _onlyif_ => "match service-name[port = '12345'][protocol = 'tcp'] size == 1", } This would avoid throwing any errors but won't set anything unless there is already another service using the desired port and protocol (then it will just replace the service name). It all depends on the user's scope. For more information about puppetlabs/puppetlabs-augeas_core usage and examples, please check https://puppet.com/docs/puppet/5.5/resources_augeas.html.   A final note regarding the desired behaviour that might be important mentioning: augeas doesn't check the ports for availability (or any other field); it simply parses and adds/modifies/queries data in configuration files as requested by the user using configuration file template logic from the given/available lenses (*.aug files). Add Comment   This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.340188.1576739160000.79800.1605544200136%40Atlassian.JIRA.