built-in feature of DBSight and works well in real production
sites. I am not sure how payload can help. If payload is used to store date,
it seems redundant since you can easily store date field in the document,
and you can get the date from the field cache pretty quickly.
--
Chris Lu
the problem s quickly.
--
Chris Lu
-
Instant Scalable Full-Text Search On Any Database/Application
site: http://www.dbsight.net
demo: http://search.dbsight.com
Lucene Database Search in 3 minutes:
http://wiki.dbsight.com/index.php?title
[
https://issues.apache.org/jira/browse/LUCENE-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lu updated LUCENE-1383:
-
Attachment: ScreenHunter_07 Sep. 13 19.13.jpg
this is the V shape memory graph using the patch. This
[
https://issues.apache.org/jira/browse/LUCENE-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630845#action_12630845
]
chrislusf edited comment on LUCENE-1383 at 9/13/08 7:12 PM:
---
On
[
https://issues.apache.org/jira/browse/LUCENE-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630845#action_12630845
]
Chris Lu commented on LUCENE-1383:
--
On second thought, I think this is normal beha
[
https://issues.apache.org/jira/browse/LUCENE-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630789#action_12630789
]
Chris Lu commented on LUCENE-1383:
--
The test is repeatedly close and open the i
[
https://issues.apache.org/jira/browse/LUCENE-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lu updated LUCENE-1383:
-
Attachment: ScreenHunter_02 Sep. 13 08.42.jpg
Beginning of a memory increase of 2 RAMDirectory in the
[
https://issues.apache.org/jira/browse/LUCENE-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lu updated LUCENE-1383:
-
Attachment: ScreenHunter_03 Sep. 13 08.43.jpg
Endding of a memory increase of 2 RAMDirectory in the
[
https://issues.apache.org/jira/browse/LUCENE-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lu updated LUCENE-1383:
-
Attachment: ScreenHunter_01 Sep. 13 08.40.jpg
A little bump when closing and opening the index. The
[
https://issues.apache.org/jira/browse/LUCENE-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630784#action_12630784
]
Chris Lu commented on LUCENE-1383:
--
I can confirm this patch fixed the memory pro
ually, I didn't look at the javadoc of
close(), because, shouldn't close() means close(), not uncontrollably
delayed resource releasing? So I fear just changing the javadoc is not
enough.
--
Chris Lu
-
Instant Scalable Full-Text Search On Any Database/Applica
this
file, I will have to upgrade Lucene and manually remove the patch
Lucene-1195.
--
Chris Lu
-
Instant Scalable Full-Text Search On Any Database/Application
site: http://www.dbsight.net
demo: http://search.dbsight.com
Lucene Database Search in 3 minutes:
http
Yeah, the timing is different. But it's an unknown, undetermined, and
uncontrollable time...
We can not ask the user,
while(memory is low){
sleep(1000);
}
do_the_real_thing_an_hour_later
--
Chris Lu
-
Instant Scalable Full-Text Search On Any Database/Application
SafeThreadLocal is very interesting. It'll be good not only for Lucene, but
also other projects.
Could you please post it?
--
Chris Lu
-
Instant Scalable Full-Text Search On Any Database/Application
site: http://www.dbsight.net
demo: http://search.dbsight.com
L
Not likely. Actually I made some changes to Lucene source code and I can see
the changes in the memory snapshot. So it is the latest Lucene version.
--
Chris Lu
-
Instant Scalable Full-Text Search On Any Database/Application
site: http://www.dbsight.net
demo: http
Not holding searcher/reader. I did check that via memory snapshot.
--
Chris Lu
-
Instant Scalable Full-Text Search On Any Database/Application
site: http://www.dbsight.net
demo: http://search.dbsight.com
Lucene Database Search in 3 minutes:
http://wiki.dbsight.com
? duh...
I can only say this is a problem to be cleaned up.
--
Chris Lu
-
Instant Scalable Full-Text Search On Any Database/Application
site: http://www.dbsight.net
demo: http://search.dbsight.com
Lucene Database Search in 3 minutes:
http://wiki.dbsight.com/
Is it possible that some other places that's using SegmentTermEnum as
ThreadLocal?This may explain why TermInfosReader.ThreadResources is not in
the memory snapshot.
--
Chris Lu
-
Instant Scalable Full-Text Search On Any Database/Application
site: http://www.dbsigh
really increase, since the IndexReader/enums/etc are
> not very large...
>
> When you create a new RAMDirectories, you need to BE CERTAIN !!! that the
> other IndexReaders/Searchers using the old RAMDirectory are ALL CLOSED,
> otherwise their memory will still be in use, which leads to
ming much index updates. if we only create the index and
don't do nothing except searching it, it work fine.
-------
--
Chris Lu
-
Instant Scalable Full-Text Search On Any Database/Applicatio
ne.
maybe you can give me a hint or a link,
greetz,
-
--
Chris Lu
-
Instant Scalable Full-Text Search On Any Database/Application
site: http://www.dbsight.net
demo: http://search.dbsight.com
Lucene Database Search in 3 min
Does this make any difference?If I intentionally close the searcher and
reader failed to release the memory, I can not rely on some magic of JVM to
release it.
--
Chris Lu
-
Instant Scalable Full-Text Search On Any Database/Application
site: http://www.dbsight.net
demo
Actually, even I only use one IndexReader, some resources are cached via the
ThreadLocal cache, and can not be released unless all threads do the close
action.
SegmentTermEnum itself is small, but it holds RAMDirectory along the path,
which is big.
--
Chris Lu
-
Instant
IndexReader in the
pool.
These contradictory requirements are caused by the ThreadLocal LRU cache in
the LUCENE-1195.
My only solution is to revert back this particular patch.
--
Chris Lu
-
Instant Scalable Full-Text Search On Any Database/Application
site: http://www.dbsight.net
I have tried to create an IndexReader pool and dynamically create searcher.
But the memory leak is the same. It's not related to the Searcher class
specifically, but the SegmentTermEnum in TermInfosReader.
--
Chris Lu
-
Instant Scalable Full-Text Search On Any Dat
On J2EE environment, usually there is a searcher pool with several searchers
open.The speed to opening a large index for every user is not acceptable.
--
Chris Lu
-
Instant Scalable Full-Text Search On Any Database/Application
site: http://www.dbsight.net
demo: http
recommended for J2EE environment, this has to be fixed.
--
Chris Lu
-
Instant Scalable Full-Text Search On Any Database/Application
site: http://www.dbsight.net
demo: http://search.dbsight.com
Lucene Database Search in 3 minutes:
http://wiki.dbsight.com/
er thread, the RAMDirectory
totally disappeared from the memory snapshot.
--
Chris Lu
-
Instant Scalable Full-Text Search On Any Database/Application
site: http://www.dbsight.net
demo: http://search.dbsight.com
Lucene Database Search in 3 minutes:
http
n 659601, right before this patch is
checked in, the memory leak is gone.
Although my case is RAMDirectory, I believe this will affect disk based
index also.
--
Chris Lu
-
Instant Scalable Full-Text Search On Any Database/Application
site: http://www.dbsight.net
demo: http://sea
29 matches
Mail list logo
