Re: monitoring ruby app memory usage with passenger-recycler (was: Cannot require library for passenger in ruby)
___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: monitoring ruby app memory usage with passenger-recycler (was: Cannot require library for passenger in ruby)
On Fri, 23 Feb 2018, Karli Sjöberg wrote: > On Thu, 2018-02-22 at 18:28 +, Marcin Cieslak wrote: > > > than > > > nothing. It now runs and does what you´d expect :) > > > > > > I have attached it to this email. > > Thanks! Now we should try to integrate it into the passenger itself > > :) > > No it´s just 'foreman_maintain' that needs fixing. I´m actually d > oing the git dance right now getting it accepted upstream :) > > https://github.com/theforeman/foreman_maintain/pull/146 Wish you luck :) I have pushed https://github.com/theforeman/foreman_maintain/pull/143 a bit earlier. (I still think that most of your code should go into passenger, but some refactoring might be needed there first). Marcin smime.p7s Description: S/MIME Cryptographic Signature
Re: monitoring ruby app memory usage with passenger-recycler (was: Cannot require library for passenger in ruby)
On Thu, 2018-02-22 at 18:28 +, Marcin Cieslak wrote: > On Thu, 22 Feb 2018, Karli Sjöberg wrote: > > > > > > > > > How do I interpret the output of 'procstat -v ' though? Can > > > I > > > sum > > > up all of the RES or PRES numbers to get the total virtual memory > > > that > > > this process is consuming? > > I hacked at it a bit and came up with an updated patch that > > includes > > the changes you made, plus my own. > > > > Since "private dirty RSS" isn´t available, I´ve used "maximum RSS" > > from > > 'procstat -r ', which is far from perfect, but it´s better > > than > > nothing. It now runs and does what you´d expect :) > > > > I have attached it to this email. > Thanks! Now we should try to integrate it into the passenger itself > :) No it´s just 'foreman_maintain' that needs fixing. I´m actually d oing the git dance right now getting it accepted upstream :) https://github.com/theforeman/foreman_maintain/pull/146 /K > > I think you might be looking for "private resident pages" (PRES), > but I am not sure. > > For your particular problem I would log procstat -v output in its > entirety > at regular intervals and see where the memory grows. > > Happy you got there somehow :) > > Marcin ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: monitoring ruby app memory usage with passenger-recycler (was: Cannot require library for passenger in ruby)
On Thu, 22 Feb 2018, Karli Sjöberg wrote: > > How do I interpret the output of 'procstat -v ' though? Can I > > sum > > up all of the RES or PRES numbers to get the total virtual memory > > that > > this process is consuming? > > I hacked at it a bit and came up with an updated patch that includes > the changes you made, plus my own. > > Since "private dirty RSS" isn´t available, I´ve used "maximum RSS" from > 'procstat -r ', which is far from perfect, but it´s better than > nothing. It now runs and does what you´d expect :) > > I have attached it to this email. Thanks! Now we should try to integrate it into the passenger itself :) I think you might be looking for "private resident pages" (PRES), but I am not sure. For your particular problem I would log procstat -v output in its entirety at regular intervals and see where the memory grows. Happy you got there somehow :) Marcin smime.p7s Description: S/MIME Cryptographic Signature
Re: monitoring ruby app memory usage with passenger-recycler (was: Cannot require library for passenger in ruby)
On Thu, 2018-02-22 at 10:11 +0100, Karli Sjöberg via freebsd-ports wrote: > On Thu, 2018-02-22 at 07:50 +0100, Karli Sjöberg via freebsd-ports > wrote: > > On Tue, 2018-02-20 at 21:31 +, Marcin Cieslak wrote: > > > On Mon, 19 Feb 2018, Karli Sjöberg wrote: > > > > > > > > What is the tool you are trying to deploy? foreman_maintain? > > > > > > Sorry, I was too quick. This foreman_main is used to > > > download/update > > > foreman > > > from RedHat Satellite servers, which is not what you want. > > > > Well, I want the "foreman_maintain" package, which includes > > "passenger- > > recycler", so I guess I do want it :) > > > > > > > > However, the passenger-recycler script is a standalone script > > > which > > > could possibly used with this little fix: > > > > Ooh, awesome! Yeah, should´ve figured since it´s only at 0.1.3 :) > > > > > > > > https://github.com/theforeman/foreman_maintain/pull/143 > > > > > > To apply this, please add files/patch-bin_passenger-recycler in > > > the > > > port I've sent before: > > > > > > --- bin/passenger-recycler.orig 2018-02-20 21:09:18 UTC > > > +++ bin/passenger-recycler > > > @@ -41,10 +41,10 @@ def process_status?(pid) > > > end > > > > > > require 'phusion_passenger' > > > +PhusionPassenger.locate_directories > > > require 'phusion_passenger/platform_info' > > > require 'phusion_passenger/platform_info/ruby' > > > require 'phusion_passenger/admin_tools/memory_stats' > > > -PhusionPassenger.locate_directories > > > stats = PhusionPassenger::AdminTools::MemoryStats.new > > > unless stats.platform_provides_private_dirty_rss_information? > > >puts 'Please run as root or platform unsupported' > > > > Yes, this works, thanks! > > > > > > > > > > > But the real problem is that Passenger does think it supports > > > getting process private dirty RSS memory information only on > > > Linux: > > > > > > https://github.com/phusion/passenger/blob/219ad24159ae4033a342e6a > > > d7 > > > 53 > > > cfee05d98bae0/src/ruby_supportlib/phusion_passenger/admin_tools/m > > > em > > > or > > > y_stats.rb#L121 > > > > > > In file > > > src/ruby_supportlib/phusion_passenger/admin_tools/memory_stats.rb > > > : > > > > > > 120 def platform_provides_private_dirty_rss_information? > > > 121 return os_name_simple == "linux" > > > 122 end > > > > > > This support probably would need to be written. So, > > > unfortunately, > > > passenger-recycler > > > as written will exit with: > > > > > > Please run as root or platform unsupported > > > > Yes, I have now gotten to this stage at least, thank you very much! > > I > > don´t mind having a look at it, it might be simplest thing to > > write, > > or > > it´s hell, but I´ll have a look! > > > > Thank you for pointing me in the right direction! > > > > /K > > > > > > > > But passenger-recycler does something very simple - if the amount > > > of > > > "private resident RSS > > > memory" is higher than specified amount in the configuration > > > file, > > > it > > > kills the process. > > > > > > You might get a much better monitoring on FreeBSD by monitoring > > > the > > > output of > > > > > > procstat -v > > > > > > which will give you details about the memory usage of the process > > > and > > > see how it is growing > > > over time. Maybe you can pinpoint the shared library that causes > > > this, if you are lucky. > > How do I interpret the output of 'procstat -v ' though? Can I > sum > up all of the RES or PRES numbers to get the total virtual memory > that > this process is consuming? I hacked at it a bit and came up with an updated patch that includes the changes you made, plus my own. Since "private dirty RSS" isn´t available, I´ve used "maximum RSS" from 'procstat -r ', which is far from perfect, but it´s better than nothing. It now runs and does what you´d expect :) I have attached it to this email. /K > /K Marcin--- bin/passenger-recycler 2018-02-22 14:19:40.0 +0100 +++ bin/passenger-recycler 2018-02-22 14:11:30.034911000 +0100 @@ -41,41 +41,76 @@ end require 'phusion_passenger'
Re: monitoring ruby app memory usage with passenger-recycler (was: Cannot require library for passenger in ruby)
On Thu, 2018-02-22 at 07:50 +0100, Karli Sjöberg via freebsd-ports wrote: > On Tue, 2018-02-20 at 21:31 +, Marcin Cieslak wrote: > > On Mon, 19 Feb 2018, Karli Sjöberg wrote: > > > > > > What is the tool you are trying to deploy? foreman_maintain? > > > > Sorry, I was too quick. This foreman_main is used to > > download/update > > foreman > > from RedHat Satellite servers, which is not what you want. > > Well, I want the "foreman_maintain" package, which includes > "passenger- > recycler", so I guess I do want it :) > > > > > However, the passenger-recycler script is a standalone script which > > could possibly used with this little fix: > > Ooh, awesome! Yeah, should´ve figured since it´s only at 0.1.3 :) > > > > > https://github.com/theforeman/foreman_maintain/pull/143 > > > > To apply this, please add files/patch-bin_passenger-recycler in the > > port I've sent before: > > > > --- bin/passenger-recycler.orig 2018-02-20 21:09:18 UTC > > +++ bin/passenger-recycler > > @@ -41,10 +41,10 @@ def process_status?(pid) > > end > > > > require 'phusion_passenger' > > +PhusionPassenger.locate_directories > > require 'phusion_passenger/platform_info' > > require 'phusion_passenger/platform_info/ruby' > > require 'phusion_passenger/admin_tools/memory_stats' > > -PhusionPassenger.locate_directories > > stats = PhusionPassenger::AdminTools::MemoryStats.new > > unless stats.platform_provides_private_dirty_rss_information? > >puts 'Please run as root or platform unsupported' > > Yes, this works, thanks! > > > > > > > But the real problem is that Passenger does think it supports > > getting process private dirty RSS memory information only on Linux: > > > > https://github.com/phusion/passenger/blob/219ad24159ae4033a342e6ad7 > > 53 > > cfee05d98bae0/src/ruby_supportlib/phusion_passenger/admin_tools/mem > > or > > y_stats.rb#L121 > > > > In file > > src/ruby_supportlib/phusion_passenger/admin_tools/memory_stats.rb: > > > > 120 def platform_provides_private_dirty_rss_information? > > 121 return os_name_simple == "linux" > > 122 end > > > > This support probably would need to be written. So, unfortunately, > > passenger-recycler > > as written will exit with: > > > > Please run as root or platform unsupported > > Yes, I have now gotten to this stage at least, thank you very much! I > don´t mind having a look at it, it might be simplest thing to write, > or > it´s hell, but I´ll have a look! > > Thank you for pointing me in the right direction! > > /K > > > > > But passenger-recycler does something very simple - if the amount > > of > > "private resident RSS > > memory" is higher than specified amount in the configuration file, > > it > > kills the process. > > > > You might get a much better monitoring on FreeBSD by monitoring the > > output of > > > > procstat -v > > > > which will give you details about the memory usage of the process > > and > > see how it is growing > > over time. Maybe you can pinpoint the shared library that causes > > this, if you are lucky. How do I interpret the output of 'procstat -v ' though? Can I sum up all of the RES or PRES numbers to get the total virtual memory that this process is consuming? /K > > > > Marcin signature.asc Description: This is a digitally signed message part
Re: monitoring ruby app memory usage with passenger-recycler (was: Cannot require library for passenger in ruby)
On Tue, 2018-02-20 at 21:31 +, Marcin Cieslak wrote: > On Mon, 19 Feb 2018, Karli Sjöberg wrote: > > > > What is the tool you are trying to deploy? foreman_maintain? > > Sorry, I was too quick. This foreman_main is used to download/update > foreman > from RedHat Satellite servers, which is not what you want. Well, I want the "foreman_maintain" package, which includes "passenger- recycler", so I guess I do want it :) > > However, the passenger-recycler script is a standalone script which > could possibly used with this little fix: Ooh, awesome! Yeah, should´ve figured since it´s only at 0.1.3 :) > > https://github.com/theforeman/foreman_maintain/pull/143 > > To apply this, please add files/patch-bin_passenger-recycler in the > port I've sent before: > > --- bin/passenger-recycler.orig 2018-02-20 21:09:18 UTC > +++ bin/passenger-recycler > @@ -41,10 +41,10 @@ def process_status?(pid) > end > > require 'phusion_passenger' > +PhusionPassenger.locate_directories > require 'phusion_passenger/platform_info' > require 'phusion_passenger/platform_info/ruby' > require 'phusion_passenger/admin_tools/memory_stats' > -PhusionPassenger.locate_directories > stats = PhusionPassenger::AdminTools::MemoryStats.new > unless stats.platform_provides_private_dirty_rss_information? >puts 'Please run as root or platform unsupported' Yes, this works, thanks! > > > But the real problem is that Passenger does think it supports > getting process private dirty RSS memory information only on Linux: > > https://github.com/phusion/passenger/blob/219ad24159ae4033a342e6ad753 > cfee05d98bae0/src/ruby_supportlib/phusion_passenger/admin_tools/memor > y_stats.rb#L121 > > In file > src/ruby_supportlib/phusion_passenger/admin_tools/memory_stats.rb: > > 120 def platform_provides_private_dirty_rss_information? > 121 return os_name_simple == "linux" > 122 end > > This support probably would need to be written. So, unfortunately, > passenger-recycler > as written will exit with: > > Please run as root or platform unsupported Yes, I have now gotten to this stage at least, thank you very much! I don´t mind having a look at it, it might be simplest thing to write, or it´s hell, but I´ll have a look! Thank you for pointing me in the right direction! /K > > But passenger-recycler does something very simple - if the amount of > "private resident RSS > memory" is higher than specified amount in the configuration file, it > kills the process. > > You might get a much better monitoring on FreeBSD by monitoring the > output of > > procstat -v > > which will give you details about the memory usage of the process and > see how it is growing > over time. Maybe you can pinpoint the shared library that causes > this, if you are lucky. > > Marcin signature.asc Description: This is a digitally signed message part
Re: monitoring ruby app memory usage with passenger-recycler (was: Cannot require library for passenger in ruby)
On Mon, 19 Feb 2018, Karli Sjöberg wrote: > > What is the tool you are trying to deploy? foreman_maintain? Sorry, I was too quick. This foreman_main is used to download/update foreman from RedHat Satellite servers, which is not what you want. However, the passenger-recycler script is a standalone script which could possibly used with this little fix: https://github.com/theforeman/foreman_maintain/pull/143 To apply this, please add files/patch-bin_passenger-recycler in the port I've sent before: --- bin/passenger-recycler.orig 2018-02-20 21:09:18 UTC +++ bin/passenger-recycler @@ -41,10 +41,10 @@ def process_status?(pid) end require 'phusion_passenger' +PhusionPassenger.locate_directories require 'phusion_passenger/platform_info' require 'phusion_passenger/platform_info/ruby' require 'phusion_passenger/admin_tools/memory_stats' -PhusionPassenger.locate_directories stats = PhusionPassenger::AdminTools::MemoryStats.new unless stats.platform_provides_private_dirty_rss_information? puts 'Please run as root or platform unsupported' But the real problem is that Passenger does think it supports getting process private dirty RSS memory information only on Linux: https://github.com/phusion/passenger/blob/219ad24159ae4033a342e6ad753cfee05d98bae0/src/ruby_supportlib/phusion_passenger/admin_tools/memory_stats.rb#L121 In file src/ruby_supportlib/phusion_passenger/admin_tools/memory_stats.rb: 120 def platform_provides_private_dirty_rss_information? 121 return os_name_simple == "linux" 122 end This support probably would need to be written. So, unfortunately, passenger-recycler as written will exit with: Please run as root or platform unsupported But passenger-recycler does something very simple - if the amount of "private resident RSS memory" is higher than specified amount in the configuration file, it kills the process. You might get a much better monitoring on FreeBSD by monitoring the output of procstat -v which will give you details about the memory usage of the process and see how it is growing over time. Maybe you can pinpoint the shared library that causes this, if you are lucky. Marcin smime.p7s Description: S/MIME Cryptographic Signature
