Well, I was able to stop the hemorrhaging. Have been watching it for about
an hour and the inotify list remains constant.
By using an inline Groovy script, CAS no longer needed to setup a watcher
service for the Groovy scripts. However, I lost the ability to log debug
info and compound statements seem to not be allowed in the inline model.
Fortunately my code was not so horribly complex that I was able to work it
out. For example:
AD groups are horribly disfigured (read in DN form) from the LDAP query, so
we rewrite them like so:
memberOf:
[
java.util.ArrayList
[
groovy { def groups = attributes['memberOf']; def result = [];
for ( cn in groups ) result.add( ( cn =~ /CN=([^,]+),/)[0][1] ) ; return
result; }
]
]
If anyone has an idea on where to begin looking for the cause of this
issue, I am quite happy to help determine why this flies out of control.
Also, if anyone has info on the syntax limitations of inline Groovy scripts
and how to still do logging in the inline script that would be very helpful
for debugging.
Thank you!
Bill
On Monday, June 1, 2020 at 12:44:23 PM UTC-4, William Jojo wrote:
>
> Been running 6.1.6 for about 2 weeks. No issues - until I added SAML
> support. This morning I noticed CAS no longer working. Checked log and
> found:
>
> From log:
>
> 2020-06-01 09:05:32,086 INFO [org.apereo.cas.util.io.PathWatcherService] -
> <*Watching directory at [/etc/cas/saml]*>
> 2020-06-01 09:05:32,086 ERROR
> [org.apereo.cas.services.ReturnMappedAttributeReleasePolicy] - <*User
> limit of inotify instances reached or too many open files*>
> java.io.IOException: *User limit of inotify instances reached or too many
> open files*
> at sun.nio.fs.LinuxWatchService.<init>(LinuxWatchService.java:64) ~[?:?]
> at sun.nio.fs.LinuxFileSystem.newWatchService(LinuxFileSystem.java:47)
> ~[?:?]
> at
> org.apereo.cas.util.io.PathWatcherService.<init>(PathWatcherService.java:62)
> ~[cas-server-core-util-api-6.1.6.jar:6.1.6]
> at
> org.apereo.cas.util.io.PathWatcherService.<init>(PathWatcherService.java:40)
> ~[cas-server-core-util-api-6.1.6.jar:6.1.6]
> at
> org.apereo.cas.util.io.FileWatcherService.<init>(FileWatcherService.java:26)
> ~[cas-server-core-util-api-6.1.6.jar:6.1.6]
> at
> org.apereo.cas.util.scripting.*WatchableGroovyScriptResource*.<init>(WatchableGroovyScriptResource.java:31)
>
> ~[cas-server-core-util-api-6.1.6.jar:6.1.6]
>
>
> Thought this was odd since never had this problem with any other area of
> CAS watch areas. Did some digging and seems this is NOT an issue UNTIL I
> added the groovy files to a SAML service.
>
> The portion of the JSON is as follows:
>
> memberOf:
> [
> java.util.ArrayList
> [
> file:/etc/cas/saml/memberOf.groovy
> ]
> ]
> eduPersonPrimaryAffiliation:
> [
> java.util.ArrayList
> [
> file:/etc/cas/saml/eduPersonPrimaryAffiliation.groovy
> ]
> ]
>
> Now look at this output:
>
> root@casdev-master:~# while (( 1 == 1 )); do date; lsof | grep inotify |
> grep 31744 | wc -l; sleep 120; done
>
> Mon Jun 1 11:28:05 EDT 2020
>
> 178
>
> Mon Jun 1 11:30:05 EDT 2020
>
> 178
>
> Mon Jun 1 11:32:06 EDT 2020
>
> 178
>
> Mon Jun 1 11:34:06 EDT 2020
>
> 178
>
> Mon Jun 1 11:36:07 EDT 2020
>
> 178
>
> Mon Jun 1 11:38:08 EDT 2020
>
> 178
>
> Mon Jun 1 11:40:08 EDT 2020
>
> 1872
>
> Mon Jun 1 11:42:09 EDT 2020
>
> 2500
>
> Mon Jun 1 11:44:10 EDT 2020
>
> 3192
>
> Mon Jun 1 11:46:11 EDT 2020
>
> 3948
>
> Mon Jun 1 11:48:12 EDT 2020
>
> 4768
>
> Mon Jun 1 11:50:13 EDT 2020
>
> 5652
>
> Mon Jun 1 11:52:14 EDT 2020
>
> 6600
>
> There are 178 inotify watches consistently UNTIL I edit the service file
> and allow the Groovy files to be used. Then it just goes out of control.
> There were this many entries for each:
>
> root@casdev-master:~# lsof | grep inotify | grep 31744 | grep edu | wc -l
> 1200
> root@casdev-master:~# lsof | grep inotify | grep 31744 | grep member | wc
> -l
> 1104
>
> It seems too be increasing by hundreds of entries per TID in a very brief
> period of time and it also seems to be affecting other inotify counts as a
> result. Any thoughts on why this would suddenly go out of control when
> adding Groovy files to the service?
>
> Thank you!
>
> Bill
>
>
--
- Website: https://apereo.github.io/cas
- Gitter Chatroom: https://gitter.im/apereo/cas
- List Guidelines: https://goo.gl/1VRrw7
- Contributions: https://goo.gl/mh7qDG
---
You received this message because you are subscribed to the Google Groups "CAS
Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/a/apereo.org/d/msgid/cas-user/33a38fa3-2d96-4fac-9e6a-1b1bd341a444%40apereo.org.
