** Description changed:
[SRU]
[Impact]
The use of memory grows as a site conf file enlarges when using augeas.
[Test Plan]
- Copy the attached conf file here to /etc/apache2/sites-available and make
the site available to apache (a2ensite memorybomb).
- Mime module has to be enabled as well (check it with a2query -m | grep
mime). If not, enabling it with a2enmod mime.
- Reload apache2 configuration ( systemctl reload apache2 ). If something
wrong here, maybe a non-graphical web browser like lynx is need to be installed.
- Open htop in a different terminal
- Run augtool -I httpd.aug
- In augtool prompt use: match
/files/etc/apache2/sites-available/*[label()=~regexp('.*\\.conf')]//*[self::directive=~regexp('([Ii][Nn][Cc][Ll][Uu][Dd][Ee])|([Ii][Nn][Cc][Ll][Uu][Dd][Ee])|([Ii][Nn][Cc][Ll][Uu][Dd][Ee][Oo][Pp][Tt][Ii][Oo][Nn][Aa][Ll])')]
- Check in htop that augtool is consuming:
- Before fixing the issue: near 2238MB
- After fixing the issue (updated packages needed are augeas-lenses,
augeas-tools and libaugeas0): near 179 MB
PPA with built package with fix for testing: ppa:mirespace/sru-
lp-1828074-bionic-augeas-large-http-files
[Where problems could occur]
This patch is cherry picked from upstream, so the bug is corrected in
- following augeas package versions.
+ following augeas package versions and no other conflicts have arisen
+ with this change.
+
+ Also, the change itself acts as a cache of evaluated expressions
+ (substituting the abstract syntax tree in memory for the value it
+ produces), so the way expressions are evaluated is the same before and
+ after the change and no other part of augeas has to be affected by this.
[Other Info]
Fixed upstream in version 1.11.0, commit id :
bbf31f719db54916993be9042254f6d77b61cb13
Commit description: Replace pure function invocations in path
expressions with their result.
In path expressions, we generally need to evaluate functions against
every node that we consider for the result set. For example, in the path
expression /files/etc/hosts/*[ipaddr =~ regexp('127\\.')], the regexp
function was evaluated against every entry in /etc/hosts. Evaluating that
function requires the construction and compilation of a new regexp.
Because of how memory is managed during evaluation of path expressions,
the memory used by all these copies of the same regexp is only freed
after we are done evaluating the path expression. This causes
unacceptable memory usage in large files (see hercules-team#569).
To avoid these issues, we now distinguish between pure and impure
functions in the path expression interpreter. When we encounter a pure
function, we change the AST for the path expression so that the function
invocation is replaced with the result of invoking the function. With
the example above, that means we only construct and compile the regexp
'127\\.' once, regardless of how many nodes it gets checked against.
That leads to a dramatic reduction in the memory required to evaluate
path expressions with such constructs against large files.
Fixes hercules-team#569
Bug on upstream: https://github.com/hercules-team/augeas/issues/569
[Original Report]
This bug in libaugeas was fixed in version 1.11.0, but is not in the 18.04.2
repo.
See: https://github.com/hercules-team/augeas/issues/569
I found it using certbot and started a bug report there.
See: https://github.com/certbot/certbot/issues/7035
Which led me here to see if the newer version could be back-ported.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1828074
Title:
Memory usage issues with large Apache httpd configuration files
To manage notifications about this bug go to:
https://bugs.launchpad.net/augeas/+bug/1828074/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
