Hello Raphael!
On Tue, Aug 23, 2011 at 12:33 PM, Raphael Valyi <rva...@gmail.com> wrote:
> The issue is some of those customizable script tend to use Ruby goodness to
> generate lot's of classes. Imagine if you call a Rails script console in
> that flow for instance, there will be class generated for your models.
Is this using class reification or some other mechanism?
> What happens is that this class generation is ending up bloating the
> PergmGen memory space, making it a memory hog and finally breaking into an
> out of memory.
Geneating lots of classes will always be fraught with peril,
especially if there's an endless stream of classes. Setting up the
class unloading flag might still be the best option. However...
> What we want is to get the generate proxies classes like OpenERPProduct,
> OpenERPCustomer (generated by the ooor gem) to persist from one
> transformation execution to an other.
It is possible to share some things, like generated bytecode/classes
that are not tied to a specific JRuby instance. We do this in some
modes for jitted code. But reified classes and other forms hold hard
references to instance-specific features, and can't be shared.
> Playing with the several LocalContextScope options, the only way I found was
> to create only one single ScriptContainer instance, that I then share (using
> a Singleton pattern) between the various Ruby Kettle steps (eg living in
> different threads). and to use the LocalContextScope.SINGLETON option. Then
> I can avoid re-generating my proxy classes because they persist from one
> scriptlet execution to another.
> The issue is that with LocalContextScope.SINGLETON, each thread is altering
> he same global variables and our several Ruby Kettle steps are very likely
> to conflict one another (this happens in my tests)!
I'm not expert in the different forms of LocalContextScope, but if
it's possible to share the classes we can probably make it happen.
> The alternative I see would be the -XX:+CMSClassUnloadingEnabled JVM option
> (I didn' test yet). But this doesn't look like the optimal solution.
> Also I wonder, if I use ruby methods from say OpenERPProduct proxy and this
> class is re-generated a each data transformation run, would the JVM JIT
> and in-lining perform properly?
Any time you're re-creating classes a lot, JIT is going to suffer.
However, depending on what classes you're creating, it may not matter;
reified classes are only used as the Java object in memory, and all
code still runs through the same jit process, reusing already-jitted
code and allowing the JVM to optimize well.
The CMS option is a good one to try. It's unfortunate that by default
CMS does not unload classes.
> As far as I can see, the issue is a bit the same as with jruby-rack gem. I
> jruby-rack I see that you are using a thread of Ruby runtimes and that you
> are not using Redbridge at all.
> Would there be a solution with Redbridge for that kind of issue, what would
> you advise?
jruby-rack does set up a cross-instance cache of jitted code, but
jitted code usually isn't something that grows without bounds either.
It doesn't do anything to share reified classes, for various reasons
(such as the per-instance data in them).
- Charlie
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
