I should have used the word "encapsulate" instead of "store". :-)
On 6/8/07, Briggs <[EMAIL PROTECTED]> wrote:
> Well, you could always 'freeze' it, just create a decorator for it. So,
> create a new Configuration (call it ImmutableConfiguration) store the
> original configuration object in it,
Well, you could always 'freeze' it, just create a decorator for it. So,
create a new Configuration (call it ImmutableConfiguration) store the
original configuration object in it, and delegate the methods appropriately.
Wouldn't that work?
On 6/8/07, Doğacan Güney <[EMAIL PROTECTED]> wrote:
On 5/31/07, Nicolás Lichtmaier <[EMAIL PROTECTED]> wrote:
>
> > Actually thinking a bit further into this, I kind of agree with you. I
> > initially thought that the best approach would be to change
> > PluginRepository.get(Configuration) to PluginRepository.get() where
> > get() just creates a con
> Actually thinking a bit further into this, I kind of agree with you. I
> initially thought that the best approach would be to change
> PluginRepository.get(Configuration) to PluginRepository.get() where
> get() just creates a configuration internally and initializes itself
> with it. But then we
On 5/30/07, Doğacan Güney <[EMAIL PROTECTED]> wrote:
> On 5/30/07, Andrzej Bialecki <[EMAIL PROTECTED]> wrote:
> > Doğacan Güney wrote:
> >
> > > My patch is just a draft to see if we can create a better caching
> > > mechanism. There are definitely some rough edges there:)
> >
> > One important in
On 5/30/07, Andrzej Bialecki <[EMAIL PROTECTED]> wrote:
> Doğacan Güney wrote:
>
> > My patch is just a draft to see if we can create a better caching
> > mechanism. There are definitely some rough edges there:)
>
> One important information: in future versions of Hadoop the method
> Configuration.
Doğacan Güney wrote:
> My patch is just a draft to see if we can create a better caching
> mechanism. There are definitely some rough edges there:)
One important information: in future versions of Hadoop the method
Configuration.setObject() is deprecated and then will be removed, so we
have to
Hi,
On 5/29/07, Nicolás Lichtmaier <[EMAIL PROTECTED]> wrote:
>
> > Which job causes the problem? Perhaps, we can find out what keeps
> > creating a conf object over and over.
> >
> > Also, I have tried what you have suggested (better caching for plugin
> > repository) and it really seems to make
>> I'm having big troubles with nutch 0.9 that I hadn't with 0.8. It seems
>> that the plugin repository initializes itself all the timem until I get
>> an out of memory exception. I've been seeing the code... the plugin
>> repository mantains a map from Configuration to plugin repositories, but
>
> Which job causes the problem? Perhaps, we can find out what keeps
> creating a conf object over and over.
>
> Also, I have tried what you have suggested (better caching for plugin
> repository) and it really seems to make a difference. Can you try with
> this patch(*) to see if it solves your pr
I'll have to get around to trying this in the future. I have already
'forked' the code. But, would like to get back on track too. So,
guess I will post something, someday. The plugin part is now the
least of my worries. Again, the parsing is what is killing me now. I
don't use nutch in the 'o
On 5/29/07, Briggs <[EMAIL PROTECTED]> wrote:
> I have also noticed this. The code explicitly loads an instance of the
> plugins for every fetch (well, or parse etc., depending on what you
> are doing). This causes OutOfMemoryErrors. So, if you dump the heap,
> you can see the filter classes get lo
I have also noticed this. The code explicitly loads an instance of the
plugins for every fetch (well, or parse etc., depending on what you
are doing). This causes OutOfMemoryErrors. So, if you dump the heap,
you can see the filter classes get loaded and the never get unloaded
(they are loaded withi
Hi,
On 5/28/07, Nicolás Lichtmaier <[EMAIL PROTECTED]> wrote:
> I'm having big troubles with nutch 0.9 that I hadn't with 0.8. It seems
> that the plugin repository initializes itself all the timem until I get
> an out of memory exception. I've been seeing the code... the plugin
> repository manta
More info...
I see "map" progressing from 0% to 100. It seems to reload plugins whan
reaching 100%. Besides, I've realized that each NutchJob is a
Configuration, so (as is there's no "equals") a plugin repo would be
created per each NutchJob...
---
15 matches
Mail list logo
