[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-20 Thread Steve Kamerman
That does seem more appropriate than putting all the SAPI methods in
php-common.  The HHVM guys (Paul Tarjan, etc) gave me the green light to
do the session cleanup bit, so I'll take care of it once we settle on a
script.

Wouldn't /usr/lib/php5/sapi/sapi be a better location for scripts?

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to php5 in Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-20 Thread Steve Kamerman
Btw, if we go the route of separate sapi folders inside
/usr/lib/php5/sapi/ we should also move php5-fpm-checkconf into
/usr/lib/php5/sapi/fpm/ (I used fpm instead of php5-fpm here since
that how it is in /etc/php5/).

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to php5 in Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-20 Thread Steve Kamerman
I prefer option #1 - sourcing the sessionclean scripts from
/usr/lib/php5/sapi/* or perhaps /usr/lib/php5/sapi/sapi/sessionclean
so we can make room for more sapi-specific scripts should the need
arise.

If we keep the logic in php-common, a con is that we can't use it for HHVM (not 
that HHVM should drive the decision).
A pro for this method is that we can have 
/usr/lib/php5/sapi/sapi/sessionconfig scripts that simply spit out the params 
we need, then we feed them into the sessionclean script as you've described 
(sort of like /usr/lib/php5/maxlifetime is used now)
Another pro for this method is that we can optimize cleanup by de-duplicating 
the session.save_paths and using max(session.gc_maxlifetime).

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to php5 in Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-20 Thread Steve Kamerman
That does seem more appropriate than putting all the SAPI methods in
php-common.  The HHVM guys (Paul Tarjan, etc) gave me the green light to
do the session cleanup bit, so I'll take care of it once we settle on a
script.

Wouldn't /usr/lib/php5/sapi/sapi be a better location for scripts?

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-20 Thread Steve Kamerman
Btw, if we go the route of separate sapi folders inside
/usr/lib/php5/sapi/ we should also move php5-fpm-checkconf into
/usr/lib/php5/sapi/fpm/ (I used fpm instead of php5-fpm here since
that how it is in /etc/php5/).

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-20 Thread Steve Kamerman
I prefer option #1 - sourcing the sessionclean scripts from
/usr/lib/php5/sapi/* or perhaps /usr/lib/php5/sapi/sapi/sessionclean
so we can make room for more sapi-specific scripts should the need
arise.

If we keep the logic in php-common, a con is that we can't use it for HHVM (not 
that HHVM should drive the decision).
A pro for this method is that we can have 
/usr/lib/php5/sapi/sapi/sessionconfig scripts that simply spit out the params 
we need, then we feed them into the sessionclean script as you've described 
(sort of like /usr/lib/php5/maxlifetime is used now)
Another pro for this method is that we can optimize cleanup by de-duplicating 
the session.save_paths and using max(session.gc_maxlifetime).

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-19 Thread Steve Kamerman
Indeed, I think we've passed the 80/20 mark on this one.  I'll test your
updated one today and we should be good.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to php5 in Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-19 Thread Steve Kamerman
Issue on line 8:
  echo -e $SAPIS | \

The -e is ending up in the output (perhaps dash's echo has no -e) resulting 
in this error:
  ./sessionclean: 10: [: /etc/php5/-e: unexpected operator
I've removed the -e as our input is clean anyway.

Also, I've run this to normalize the spacing: 
  sed -i s/\t// sessionclean

Other than that everything is working well.  Thanks for the help here!



** Attachment added: sessionclean
   
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+attachment/4181650/+files/sessionclean

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to php5 in Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-19 Thread Steve Kamerman
Thanks Ondřej, no apology needed, this was indeed a constructive thread.
Sometimes us open source geeks get all too used to being in charge and
we forget how to communicate effectively.  If I'm up in your neck of the
woods some time I'll look you up and we can grab a beer :)

Also, I'll do a PR on Facebook's HHVM repo which includes an HHVM-
specific GC script (basically a non-iterative version of this one).

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to php5 in Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-19 Thread Steve Kamerman
Indeed, I think we've passed the 80/20 mark on this one.  I'll test your
updated one today and we should be good.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-19 Thread Steve Kamerman
Issue on line 8:
  echo -e $SAPIS | \

The -e is ending up in the output (perhaps dash's echo has no -e) resulting 
in this error:
  ./sessionclean: 10: [: /etc/php5/-e: unexpected operator
I've removed the -e as our input is clean anyway.

Also, I've run this to normalize the spacing: 
  sed -i s/\t// sessionclean

Other than that everything is working well.  Thanks for the help here!



** Attachment added: sessionclean
   
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+attachment/4181650/+files/sessionclean

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-19 Thread Steve Kamerman
Thanks Ondřej, no apology needed, this was indeed a constructive thread.
Sometimes us open source geeks get all too used to being in charge and
we forget how to communicate effectively.  If I'm up in your neck of the
woods some time I'll look you up and we can grab a beer :)

Also, I'll do a PR on Facebook's HHVM repo which includes an HHVM-
specific GC script (basically a non-iterative version of this one).

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-18 Thread Steve Kamerman
Thanks, this is looking better now.  Here are some notes:

1. PROC_NAMES should be defined as proc_names or used as PROC_NAMES 
(inconsistent case)
2. printf %s:%s:%s\n $save_path $gc_maxlifetime has 2 arguments but the 
definition takes 3, I would simplify it to this:
echo $save_path:$gc_maxlifetime
3. If apache2filter is installed, $proc_names will contains apache2 apache2, 
which, when passed to pidof, will return all the PIDs twice, so I've uniq'd 
them by delimiting them with \n and using sort -u | xargs pidof.
4.  I'm not entirely sure how cgi method would work, but I suppose it means 
the php command is running on the system by itself, so I've used the following 
bit to discover that pidof requires php, not php5 since that is the real 
name of the process:
php -r 'sleep(60);echo Done;' 
5. We've been using HHVM for a few years for one of our products, and I could 
easily add support for that (hhvm --php -r...), but things seem a bit out of 
order there, mainly, the php.ini is in /etc/hhvm instead of /etc/php5/hhvm

I've attached an updated script.

** Attachment added: sessionclean
   
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+attachment/4180769/+files/sessionclean

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to php5 in Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-18 Thread Steve Kamerman
I've added HHVM support and discovered another issue along the way:
session.save_path can be empty and empty string.  This is actually the
default in stock PHP and HHVM.  The PHP docs imply that an empty string
is equal to /tmp
(http://php.net/manual/en/session.configuration.php#ini.session.save-
path), although that doesn't make much sense in Windows.  Anyway, I am
now checking for that as well, so if session.save_path==, we use
/tmp.

** Attachment added: sessionclean
   
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+attachment/4181081/+files/sessionclean

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to php5 in Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-18 Thread Steve Kamerman
It seems likely that HHVM will find its way into the Debian repo, so I'm
going to see if they will set the session.save_path to /var/lib/php5:
https://github.com/hhvm/packaging/pull/67

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to php5 in Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-18 Thread Steve Kamerman
I have a feeling the CGI-mode process name is php5-cgi.  I've installed
it, but I haven't been able to get Apache 2.4 and mod_actions to play
nice with it yet, so I haven't confirmed it.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to php5 in Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-18 Thread Steve Kamerman
I noticed that you changed the default session.save_path to
/var/lib/php5/sessions, which I think is a great idea (I have no idea
what the modules/ dir is for).

In order to align the HHVM packaging with the Debian PHP packaging I
have some questions:

1. Will PHP 5.6 be the default in Ubuntu 14.10?
2. Will these updated be present in 14.04.x?
3. Will these changes be present in PHP 5.5.x?
4. Are you going to be the Debian-side maintainer for the Debian-repo HHVM 
package?

The note to see /usr/lib/php5/maxlifetime is no longer needed as it doesn't 
exist
http://anonscm.debian.org/cgit/pkg-php/php.git/tree/debian/php5-common.php5.cron.d?h=master-5.6

I was able to get php-cgi working, and it uses the binary named php5
(assuming the user follows the example in /etc/apache2/conf-available
/serve-cgi-bin.conf).  Since the name php5 is ambiguous with
/usr/bin/php5, we can pass the full path to the executable, /usr/lib
/cgi-bin/php5, to pidof.

This latest script builds on the last one, but is over 2x faster by
avoiding multiple calls to PHP per SAPI.  Instead, it uses
foreach(ini_get_all(session)) to get all the session config options,
then uses simple commands to parse config options out.  It seems like
this is a little overkill, but I think it makes the code a bit more
readable, and it's a lot less expensive since PHP doesn't need to start
up with all of its modules 3 times for each SAPI.

** Attachment added: sessionclean
   
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+attachment/4181227/+files/sessionclean

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to php5 in Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-18 Thread Steve Kamerman
Thanks, this is looking better now.  Here are some notes:

1. PROC_NAMES should be defined as proc_names or used as PROC_NAMES 
(inconsistent case)
2. printf %s:%s:%s\n $save_path $gc_maxlifetime has 2 arguments but the 
definition takes 3, I would simplify it to this:
echo $save_path:$gc_maxlifetime
3. If apache2filter is installed, $proc_names will contains apache2 apache2, 
which, when passed to pidof, will return all the PIDs twice, so I've uniq'd 
them by delimiting them with \n and using sort -u | xargs pidof.
4.  I'm not entirely sure how cgi method would work, but I suppose it means 
the php command is running on the system by itself, so I've used the following 
bit to discover that pidof requires php, not php5 since that is the real 
name of the process:
php -r 'sleep(60);echo Done;' 
5. We've been using HHVM for a few years for one of our products, and I could 
easily add support for that (hhvm --php -r...), but things seem a bit out of 
order there, mainly, the php.ini is in /etc/hhvm instead of /etc/php5/hhvm

I've attached an updated script.

** Attachment added: sessionclean
   
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+attachment/4180769/+files/sessionclean

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-18 Thread Steve Kamerman
I've added HHVM support and discovered another issue along the way:
session.save_path can be empty and empty string.  This is actually the
default in stock PHP and HHVM.  The PHP docs imply that an empty string
is equal to /tmp
(http://php.net/manual/en/session.configuration.php#ini.session.save-
path), although that doesn't make much sense in Windows.  Anyway, I am
now checking for that as well, so if session.save_path==, we use
/tmp.

** Attachment added: sessionclean
   
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+attachment/4181081/+files/sessionclean

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-18 Thread Steve Kamerman
It seems likely that HHVM will find its way into the Debian repo, so I'm
going to see if they will set the session.save_path to /var/lib/php5:
https://github.com/hhvm/packaging/pull/67

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-18 Thread Steve Kamerman
I have a feeling the CGI-mode process name is php5-cgi.  I've installed
it, but I haven't been able to get Apache 2.4 and mod_actions to play
nice with it yet, so I haven't confirmed it.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-18 Thread Steve Kamerman
I noticed that you changed the default session.save_path to
/var/lib/php5/sessions, which I think is a great idea (I have no idea
what the modules/ dir is for).

In order to align the HHVM packaging with the Debian PHP packaging I
have some questions:

1. Will PHP 5.6 be the default in Ubuntu 14.10?
2. Will these updated be present in 14.04.x?
3. Will these changes be present in PHP 5.5.x?
4. Are you going to be the Debian-side maintainer for the Debian-repo HHVM 
package?

The note to see /usr/lib/php5/maxlifetime is no longer needed as it doesn't 
exist
http://anonscm.debian.org/cgit/pkg-php/php.git/tree/debian/php5-common.php5.cron.d?h=master-5.6

I was able to get php-cgi working, and it uses the binary named php5
(assuming the user follows the example in /etc/apache2/conf-available
/serve-cgi-bin.conf).  Since the name php5 is ambiguous with
/usr/bin/php5, we can pass the full path to the executable, /usr/lib
/cgi-bin/php5, to pidof.

This latest script builds on the last one, but is over 2x faster by
avoiding multiple calls to PHP per SAPI.  Instead, it uses
foreach(ini_get_all(session)) to get all the session config options,
then uses simple commands to parse config options out.  It seems like
this is a little overkill, but I think it makes the code a bit more
readable, and it's a lot less expensive since PHP doesn't need to start
up with all of its modules 3 times for each SAPI.

** Attachment added: sessionclean
   
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+attachment/4181227/+files/sessionclean

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-15 Thread Steve Kamerman
Thanks Ondřej, this is a much better script, and I agree that the lack
of session.save_path support was rather annoying.  This is still not
optimal for high-performance servers, but it seems to be a good
compromise for general use.  Also, the fact that is honors SAPI-specific
settings helps.

Also, thanks for posting the thread regarding active sessions over the
gc_maxlifetime, that is a strange issue.  It would probably be more
efficient to iterate /proc/x/fd for each of the SAPIs, but this seems
like it would break easily, plus some instances of the process (php, for
example) might be indistinguishable from the CLI version.  I'll take a
crack at it this weekend for the sake of curiosity.

Good catch on the php-cli requirement - note that
/usr/lib/php5/maxlifetime also uses php-cli in that way.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to php5 in Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-15 Thread Steve Kamerman
What do you think about running this for each SAPI?  It's actually *way*
more efficient:

for pid in $(pidof $sapi); do
find /proc/$pid/fd -lname $session_save_path/* -exec touch {} \;
done

That will iterate all the PIDs of the running SAPI (we'll need to
iterate them as well), then find all the files it has open and touch
them.

This is more efficient than lsof since it looks only at the files open
by the SAPI in question, instead of every process.  As an example,
here's the timing of my proposed way and the current way:

kamermans@steve-ubuntu:~$ time for pid in $(pidof apache2); do sudo find 
/proc/$pid/fd -lname /var/lib/php5/* -print -exec touch {} \; ; done
/proc/4202/fd/21

real0m0.143s
user0m0.069s
sys 0m0.077s

kamermans@steve-ubuntu:~$ time sudo /usr/bin/lsof -w -l +d /var/lib/php5
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
apache2 4202   33   21uW  REG8,30 277620 
/var/lib/php5/sess_bbn8sh6ljim484c4qt3inubhr3

real0m1.079s
user0m0.514s
sys 0m0.845s


Notice that my version returns /proc/4202/fd/21, but that is the file 
descriptor which is a symbolic link to 
/var/lib/php5/sess_bbn8sh6ljim484c4qt3inubhr3.  Touching the link touches the 
file, so the effect is the same.  Also, since the find command finds and 
touches the file in (almost) one operation, so it's unlikely the file will 
disappear between the time that it's found and the time that it's touched.

Yet another option is to alter the lsof call to restrict it to just
processes that match the given name(s).  This is also fast, although it
not quite as fast as the aforementioned /proc method in my test:

kamermans@steve-ubuntu:~$ time sudo /usr/bin/lsof -w -l -a +d /var/lib/php5 -c 
apache2
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
apache2 4202   33   21uW  REG8,30 266585 
/var/lib/php5/sess_18and1t1ej6frr9kjai3m1dbj4

real0m0.182s
user0m0.074s
sys 0m0.091s

Note that I added -a to AND the directory and command constraint,
otherwise they would be OR'd.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to php5 in Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-15 Thread Steve Kamerman
It just occurred to me that I was timing sudo in my last post, so here
are the sudo-less numbers:

root@steve-ubuntu:~# time for pid in $(pidof apache2); do find /proc/$pid/fd 
-lname /var/lib/php5/* -print -exec touch {} \; ; done
/proc/30005/fd/21

real0m0.042s
user0m0.016s
sys 0m0.026s


root@steve-ubuntu:~# time /usr/bin/lsof -w -l +d /var/lib/php5
COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
apache2 30005   33   21uW  REG8,30 268283 
/var/lib/php5/sess_r5mhepbrhgmgsdnvkj72ppoc37

real0m1.076s
user0m0.517s
sys 0m0.846s


root@steve-ubuntu:~# time /usr/bin/lsof -w -l -a +d /var/lib/php5 -c apache2
COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
apache2 30005   33   21uW  REG8,30 268283 
/var/lib/php5/sess_r5mhepbrhgmgsdnvkj72ppoc37

real0m0.070s
user0m0.032s
sys 0m0.052s

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to php5 in Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-15 Thread Steve Kamerman
Ok, sorry for the repeated spamming tonight.

One problem with the solution you linked to is that it is extracting the
gx_maxlifetime in seconds, then putting that in find's -cmin which
expects minutes.

I've attached a script that incorporates the improvements of SAPI-
specific session file cleaning, as well as my improved open-file
touching code.

Here's the difference in time on one of my production machines:

# pidof php5-fpm | xargs php -r 'echo $argc running threads\n;'
102 running threads

# time /usr/lib/php5/sessionclean /var/lib/php5 $(/usr/lib/php5/maxlifetime)
real0m10.631s
user0m7.876s
sys 0m2.738s


# time /tmp/sessionclean_proposed.sh
real0m0.341s
user0m0.126s
sys 0m0.241s


** Attachment added: sessionclean_propsed.sh
   
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+attachment/4178920/+files/sessionclean_propsed.sh

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to php5 in Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-15 Thread Steve Kamerman
Thanks Ondřej, this is a much better script, and I agree that the lack
of session.save_path support was rather annoying.  This is still not
optimal for high-performance servers, but it seems to be a good
compromise for general use.  Also, the fact that is honors SAPI-specific
settings helps.

Also, thanks for posting the thread regarding active sessions over the
gc_maxlifetime, that is a strange issue.  It would probably be more
efficient to iterate /proc/x/fd for each of the SAPIs, but this seems
like it would break easily, plus some instances of the process (php, for
example) might be indistinguishable from the CLI version.  I'll take a
crack at it this weekend for the sake of curiosity.

Good catch on the php-cli requirement - note that
/usr/lib/php5/maxlifetime also uses php-cli in that way.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-15 Thread Steve Kamerman
What do you think about running this for each SAPI?  It's actually *way*
more efficient:

for pid in $(pidof $sapi); do
find /proc/$pid/fd -lname $session_save_path/* -exec touch {} \;
done

That will iterate all the PIDs of the running SAPI (we'll need to
iterate them as well), then find all the files it has open and touch
them.

This is more efficient than lsof since it looks only at the files open
by the SAPI in question, instead of every process.  As an example,
here's the timing of my proposed way and the current way:

kamermans@steve-ubuntu:~$ time for pid in $(pidof apache2); do sudo find 
/proc/$pid/fd -lname /var/lib/php5/* -print -exec touch {} \; ; done
/proc/4202/fd/21

real0m0.143s
user0m0.069s
sys 0m0.077s

kamermans@steve-ubuntu:~$ time sudo /usr/bin/lsof -w -l +d /var/lib/php5
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
apache2 4202   33   21uW  REG8,30 277620 
/var/lib/php5/sess_bbn8sh6ljim484c4qt3inubhr3

real0m1.079s
user0m0.514s
sys 0m0.845s


Notice that my version returns /proc/4202/fd/21, but that is the file 
descriptor which is a symbolic link to 
/var/lib/php5/sess_bbn8sh6ljim484c4qt3inubhr3.  Touching the link touches the 
file, so the effect is the same.  Also, since the find command finds and 
touches the file in (almost) one operation, so it's unlikely the file will 
disappear between the time that it's found and the time that it's touched.

Yet another option is to alter the lsof call to restrict it to just
processes that match the given name(s).  This is also fast, although it
not quite as fast as the aforementioned /proc method in my test:

kamermans@steve-ubuntu:~$ time sudo /usr/bin/lsof -w -l -a +d /var/lib/php5 -c 
apache2
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
apache2 4202   33   21uW  REG8,30 266585 
/var/lib/php5/sess_18and1t1ej6frr9kjai3m1dbj4

real0m0.182s
user0m0.074s
sys 0m0.091s

Note that I added -a to AND the directory and command constraint,
otherwise they would be OR'd.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-15 Thread Steve Kamerman
It just occurred to me that I was timing sudo in my last post, so here
are the sudo-less numbers:

root@steve-ubuntu:~# time for pid in $(pidof apache2); do find /proc/$pid/fd 
-lname /var/lib/php5/* -print -exec touch {} \; ; done
/proc/30005/fd/21

real0m0.042s
user0m0.016s
sys 0m0.026s


root@steve-ubuntu:~# time /usr/bin/lsof -w -l +d /var/lib/php5
COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
apache2 30005   33   21uW  REG8,30 268283 
/var/lib/php5/sess_r5mhepbrhgmgsdnvkj72ppoc37

real0m1.076s
user0m0.517s
sys 0m0.846s


root@steve-ubuntu:~# time /usr/bin/lsof -w -l -a +d /var/lib/php5 -c apache2
COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
apache2 30005   33   21uW  REG8,30 268283 
/var/lib/php5/sess_r5mhepbrhgmgsdnvkj72ppoc37

real0m0.070s
user0m0.032s
sys 0m0.052s

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-15 Thread Steve Kamerman
Ok, sorry for the repeated spamming tonight.

One problem with the solution you linked to is that it is extracting the
gx_maxlifetime in seconds, then putting that in find's -cmin which
expects minutes.

I've attached a script that incorporates the improvements of SAPI-
specific session file cleaning, as well as my improved open-file
touching code.

Here's the difference in time on one of my production machines:

# pidof php5-fpm | xargs php -r 'echo $argc running threads\n;'
102 running threads

# time /usr/lib/php5/sessionclean /var/lib/php5 $(/usr/lib/php5/maxlifetime)
real0m10.631s
user0m7.876s
sys 0m2.738s


# time /tmp/sessionclean_proposed.sh
real0m0.341s
user0m0.126s
sys 0m0.241s


** Attachment added: sessionclean_propsed.sh
   
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+attachment/4178920/+files/sessionclean_propsed.sh

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-13 Thread Steve Kamerman
Thanks for the completely non-constructive message.  Google me if you
want.  Anyway, my point (and apparently your subpoint) is that the
current script is not safe for production use.  My improvements make it
slightly more suitable out of the box.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to php5 in Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-13 Thread Steve Kamerman
You can improve things even when not using the argument from
authority.  I totally agree, I was just trying to make a point that I
know what I'm talking about, as opposed to being a passive observer.
You also know what you're talking about, and I can respect that as well.

If you seriously think running lsof on a server every 30 minutes
regardless of the necessity is perfectly safe, then I guess we can agree
to disagree :)

Prior to this bug report, my impression was that Ubuntu is developed in
an open fashion where everybody is welcome to contribute.
(http://community.ubuntu.com/contribute/developers/)  This incident,
however, has put a bad taste in my mouth.  Had I not been an open-source
developer already, I would certainly have turned away from the open-
source community.

Anyway, it is obvious that you are not going to consider my
contribution, so I will stop pushing the issue.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to php5 in Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-13 Thread Steve Kamerman
Thanks for the completely non-constructive message.  Google me if you
want.  Anyway, my point (and apparently your subpoint) is that the
current script is not safe for production use.  My improvements make it
slightly more suitable out of the box.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-13 Thread Steve Kamerman
You can improve things even when not using the argument from
authority.  I totally agree, I was just trying to make a point that I
know what I'm talking about, as opposed to being a passive observer.
You also know what you're talking about, and I can respect that as well.

If you seriously think running lsof on a server every 30 minutes
regardless of the necessity is perfectly safe, then I guess we can agree
to disagree :)

Prior to this bug report, my impression was that Ubuntu is developed in
an open fashion where everybody is welcome to contribute.
(http://community.ubuntu.com/contribute/developers/)  This incident,
however, has put a bad taste in my mouth.  Had I not been an open-source
developer already, I would certainly have turned away from the open-
source community.

Anyway, it is obvious that you are not going to consider my
contribution, so I will stop pushing the issue.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs


[Bug 1356113] [NEW] PHP5 session clean cron job causes OOM

2014-08-12 Thread Steve Kamerman
Public bug reported:

The php5 file-based session cleaner script /var/lib/php5/sessionclean is
extremely expensive, taking over 10 seconds to run on my laptop, even
when no PHP file-based session storage is enabled and the system is
idle.  My servers, on the other hand, are completely overwhelmed by this
script, which runs every 30 minutes by default.

The script performs an lsof +d /var/lib/php5, which can itself take
several minutes on my servers when they serving peak traffic of about
500 req/sec.  This lsof command is extremely expensive (imagine
iterating 5000 threads/procs, each of which has dozens or hundreds of
open files).  As a result of the load of this command, the server's
connections back up leading to an out of memory condition (OOM) at which
point the kernel tells oom_killer to start killing processes.

The really kicker here is that my session management is not done with
files (most high-performance PHP sites would not be), so all this work
is done in vain.

As a subject-matter expert in PHP and GNU/Linux, I recommend we update 
/usr/lib/php5/sessionclean and add /usr/lib/php5/sessionstorage 
New versions: https://gist.github.com/kamermans/27218b1a6dd61756b647

The new sessionstorage script simply prints out which
session.save_handlers are enabled in the installed PHP versions (not
including the cli).  The improved sessionclean script uses the
sessionstorage to run only if the files session.save_handler is
enabled, thus removing the lsof burden on most high-performance servers.
Note that I am doing some creative scripting since Bash-style arrays are
not available in Dash.

For reference, I am running on AWS c3.xlarge instances with NGINX,
PHP5-FPM and Redis (no MySQL, no sessions at all).  sessionclean kills
my instances when the total CPU usage is around 20% (where 100% is a
load of 4.0 on my 4-vCPU server), so I have a lot more room for requests
if this is fixed.

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: php5-common 5.5.9+dfsg-1ubuntu4.3
ProcVersionSignature: Ubuntu 3.13.0-29.53-generic 3.13.11.2
Uname: Linux 3.13.0-29-generic x86_64
ApportVersion: 2.14.1-0ubuntu3.3
Architecture: amd64
Date: Tue Aug 12 23:46:17 2014
Ec2AMI: ami-62a2730a
Ec2AMIManifest: (unknown)
Ec2AvailabilityZone: us-east-1c
Ec2InstanceType: c3.xlarge
Ec2Kernel: aki-919dcaf8
Ec2Ramdisk: unavailable
ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=set
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: php5
UpgradeStatus: Upgraded to trusty on 2014-07-30 (13 days ago)
modified.conffile..etc.cron.d.php5: [modified]
mtime.conffile..etc.cron.d.php5: 2014-08-12T20:02:50.878339

** Affects: php5 (Ubuntu)
 Importance: Undecided
 Status: New


** Tags: amd64 apport-bug ec2-images trusty

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to php5 in Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-12 Thread Steve Kamerman
On another note, why would we possibly need to run lsof every time we
need to cleanup sessions?  This is a very inefficient and expensive
process.  If someone knows how it came to be that this code was added, I
would be very curious.  Also, I see that recent versions of Ubuntu
(14.04 for example) put a modules/ folder in /var/lib/php which is quite
disconcerting since there are potentially thousands of session files in
there as well.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to php5 in Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 1356113] [NEW] PHP5 session clean cron job causes OOM

2014-08-12 Thread Steve Kamerman
Public bug reported:

The php5 file-based session cleaner script /var/lib/php5/sessionclean is
extremely expensive, taking over 10 seconds to run on my laptop, even
when no PHP file-based session storage is enabled and the system is
idle.  My servers, on the other hand, are completely overwhelmed by this
script, which runs every 30 minutes by default.

The script performs an lsof +d /var/lib/php5, which can itself take
several minutes on my servers when they serving peak traffic of about
500 req/sec.  This lsof command is extremely expensive (imagine
iterating 5000 threads/procs, each of which has dozens or hundreds of
open files).  As a result of the load of this command, the server's
connections back up leading to an out of memory condition (OOM) at which
point the kernel tells oom_killer to start killing processes.

The really kicker here is that my session management is not done with
files (most high-performance PHP sites would not be), so all this work
is done in vain.

As a subject-matter expert in PHP and GNU/Linux, I recommend we update 
/usr/lib/php5/sessionclean and add /usr/lib/php5/sessionstorage 
New versions: https://gist.github.com/kamermans/27218b1a6dd61756b647

The new sessionstorage script simply prints out which
session.save_handlers are enabled in the installed PHP versions (not
including the cli).  The improved sessionclean script uses the
sessionstorage to run only if the files session.save_handler is
enabled, thus removing the lsof burden on most high-performance servers.
Note that I am doing some creative scripting since Bash-style arrays are
not available in Dash.

For reference, I am running on AWS c3.xlarge instances with NGINX,
PHP5-FPM and Redis (no MySQL, no sessions at all).  sessionclean kills
my instances when the total CPU usage is around 20% (where 100% is a
load of 4.0 on my 4-vCPU server), so I have a lot more room for requests
if this is fixed.

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: php5-common 5.5.9+dfsg-1ubuntu4.3
ProcVersionSignature: Ubuntu 3.13.0-29.53-generic 3.13.11.2
Uname: Linux 3.13.0-29-generic x86_64
ApportVersion: 2.14.1-0ubuntu3.3
Architecture: amd64
Date: Tue Aug 12 23:46:17 2014
Ec2AMI: ami-62a2730a
Ec2AMIManifest: (unknown)
Ec2AvailabilityZone: us-east-1c
Ec2InstanceType: c3.xlarge
Ec2Kernel: aki-919dcaf8
Ec2Ramdisk: unavailable
ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=set
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: php5
UpgradeStatus: Upgraded to trusty on 2014-07-30 (13 days ago)
modified.conffile..etc.cron.d.php5: [modified]
mtime.conffile..etc.cron.d.php5: 2014-08-12T20:02:50.878339

** Affects: php5 (Ubuntu)
 Importance: Undecided
 Status: New


** Tags: amd64 apport-bug ec2-images trusty

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs


[Bug 1356113] Re: PHP5 session clean cron job causes OOM

2014-08-12 Thread Steve Kamerman
On another note, why would we possibly need to run lsof every time we
need to cleanup sessions?  This is a very inefficient and expensive
process.  If someone knows how it came to be that this code was added, I
would be very curious.  Also, I see that recent versions of Ubuntu
(14.04 for example) put a modules/ folder in /var/lib/php which is quite
disconcerting since there are potentially thousands of session files in
there as well.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1356113

Title:
  PHP5 session clean cron job causes OOM

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1356113/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs


[Bug 689734] Re: HAProxy fails to start at boot time

2011-07-14 Thread Steve Kamerman
There was a fairly significant bugfix in HAProxy v 1.4.15, April 2011
that prevents DOS attacks and HAProxy crashes.  I've built a .deb for it
with the same config options as the existing version (TCP Splicing, Full
Transparent Proxy, PCRE).  Networking is listed as a startup dependency,
so it should fall after that in the startup order.  Can you test it to
make sure it's starting up properly?

http://www.stevekamerman.com/haproxy/

I've got to translate the changelog from the haproxy site into deb
format yet, but I'll submit the package to Ubuntu after that.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/689734

Title:
  HAProxy fails to start at boot time

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/haproxy/+bug/689734/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs