I'm not against that.
- Mark
http://www.lucidimagination.com (mobile)
On Oct 22, 2009, at 1:36 AM, Noble Paul നോബിള് नो
ब्ळ् <noble.p...@corp.aol.com> wrote:
On Wed, Oct 21, 2009 at 6:34 PM, Mark Miller <markrmil...@gmail.com>
wrote:
bq. and Mark is representing "just keep working, ok?".
But I'm not :) Like I said, I don't view the purpose of a soft value
cache as avoiding OOM's. Size your caches correctly for that.
For those that don't understand the how and why of soft value caches,
they probably should not choose to use it.
Users may not have a clue on how much memory eventually the caches
will take up. Now if the admin page can let them know cache trashing
has happened , they can think of adding more RAM
Lance Norskog wrote:
On-topic: Will the Google implementations + soft references behave
well with 8+ processors?
Semi-on-topic: If you want to really know multiprocessor algorithms,
this is the bible: "The Art Of Multiprocessor Programming". Hundreds
of parallel algorithms for many different jobs, all coded in Java,
and
cross-referenced with the java.util.concurrent package. Just
amazing.
http://www.elsevier.com/wps/find/bookdescription.cws_home/714091/description#description
Off-topic: I was representing a system troubleshooting philosophy:
"Fail Early, Fail Loud". Meaning, if there is a problem like OOMs,
tell me and I'll fix it permanently. But different situations call
for
different answers, and Mark is representing "just keep working,
ok?".
Brittle v.s. Supple is one way to think of it.
On Tue, Oct 20, 2009 at 11:27 AM, Shalin Shekhar Mangar
<shalinman...@gmail.com> wrote:
On Tue, Oct 20, 2009 at 3:56 PM, Mark Miller
<markrmil...@gmail.com> wrote:
On Oct 20, 2009, at 12:12 AM, Shalin Shekhar Mangar <
shalinman...@gmail.com> wrote:
I don't think the debate is about weak reference vs. soft
references.
There appears to be confusion between the two here no matter
what the
debate - soft references are for cachinh, weak references are
not so much.
Getting it right is important.
I
guess the point that Lance is making is that using such a
technique will
make application performance less predictable. There's also a
good chance
that a soft reference based cache will cause cache thrashing
and will hide
OOMs caused by inadequate cache sizes. So basically we trade an
OOM for
more
CPU usage (due to re-computation of results).
That's the whole point. Your not hiding anything. I don't follow
you.
Using a soft reference based cache can hide the fact that one has
inadequate
memory for the cache size one has configured. Don't get me wrong.
I'm not
against the feature. I was merely trying to explain Lance's
concerns as I
understood them.
Personally, I think giving an option is fine. What if the user
does not
have
enough RAM and he is willing to pay the price? Right now, there
is no way
he
can do that at all. However, the most frequent reason behind
OOMs is not
having enough RAM to create the field caches and not Solr
caches, so I'm
not
sure how important this is.
How important is any feature? You don't have a use for it, so
it's not
important to you - someone else does so it is important to them.
Soft value
caches can be useful.
Don't jump to conclusions :)
The reason behind this feature request is to have Solr caches
which resize
themselves when enough memory is not available. I agree that soft
value
caches are useful for this. All I'm saying is that most OOMs that
get
reported on the list are due to inadequate free memory for
allocating field
caches. Finding a way around that will be the key to make a
Lucene/Solr
application practical in a limited memory environment.
Just for the record, I'm +1 for adding this feature but keeping
the current
behavior as the default.
--
Regards,
Shalin Shekhar Mangar.
--
- Mark
http://www.lucidimagination.com
--
-----------------------------------------------------
Noble Paul | Principal Engineer| AOL | http://aol.com
