Is There a Way To Split The Lucene Index Segments To Samller Size Less Than 1 GB
Hi Lucene Team, If you know or if there is any way of splitting Lucene indexing segments to smaller segments of size less than 1 GB, can you please know me? Here I am giving one index segments sizes, total size of index is 9.7 GB, here there are three Lucene files a) _12r7.prx b) _kft.prx c) _ls6.prx of size greater than 1 GB. I want to split them to different pieces and want to reduce their size. [r...@sc-s1-172-1.oxford.com ~]# ls -lh /index/TP_499/ total 9.7G -rw-r--r-- 1 appuser appuser 80M Jul 27 13:53 _12r7.fdt -rw-r--r-- 1 appuser appuser 1.4M Jul 27 13:53 _12r7.fdx -rw-r--r-- 1 appuser appuser 397 Jul 27 13:53 _12r7.fnm -rw-r--r-- 1 appuser appuser 649M Jul 27 13:58 _12r7.frq -rw-r--r-- 1 appuser appuser 3.9M Jul 27 13:58 _12r7.nrm -rw-r--r-- 1 appuser appuser 2.2G Jul 27 13:58 _12r7.prx -rw-r--r-- 1 appuser appuser 33 Jul 27 13:58 _12r7.stats -rw-r--r-- 1 appuser appuser 334K Jul 27 13:58 _12r7.tii -rw-r--r-- 1 appuser appuser 28M Jul 27 13:58 _12r7.tis -rw-r--r-- 1 appuser appuser 24K Jul 27 14:44 _12ts.fdt -rw-r--r-- 1 appuser appuser 400 Jul 27 14:44 _12ts.fdx -rw-r--r-- 1 appuser appuser 361 Jul 27 14:44 _12ts.fnm -rw-r--r-- 1 appuser appuser 90K Jul 27 14:44 _12ts.frq -rw-r--r-- 1 appuser appuser 1.1K Jul 27 14:44 _12ts.nrm -rw-r--r-- 1 appuser appuser 218K Jul 27 14:44 _12ts.prx -rw-r--r-- 1 appuser appuser 25 Jul 27 14:44 _12ts.stats -rw-r--r-- 1 appuser appuser 8.7K Jul 27 14:44 _12ts.tii -rw-r--r-- 1 appuser appuser 656K Jul 27 14:44 _12ts.tis -rw-r--r-- 1 appuser appuser 309K Jul 27 14:44 _12tt.fdt -rw-r--r-- 1 appuser appuser 5.1K Jul 27 14:44 _12tt.fdx -rw-r--r-- 1 appuser appuser 361 Jul 27 14:44 _12tt.fnm -rw-r--r-- 1 appuser appuser 1.9M Jul 27 14:44 _12tt.frq -rw-r--r-- 1 appuser appuser 14K Jul 27 14:44 _12tt.nrm -rw-r--r-- 1 appuser appuser 3.7M Jul 27 14:44 _12tt.prx -rw-r--r-- 1 appuser appuser 29 Jul 27 14:44 _12tt.stats -rw-r--r-- 1 appuser appuser 38K Jul 27 14:44 _12tt.tii -rw-r--r-- 1 appuser appuser 2.6M Jul 27 14:44 _12tt.tis -rw-r--r-- 1 appuser appuser 62M Jul 15 19:51 _kft.fdt -rw-r--r-- 1 appuser appuser 1.3M Jul 15 19:51 _kft.fdx -rw-r--r-- 1 appuser appuser 397 Jul 15 19:51 _kft.fnm -rw-r--r-- 1 appuser appuser 626M Jul 15 20:40 _kft.frq -rw-r--r-- 1 appuser appuser 3.5M Jul 15 20:40 _kft.nrm -rw-r--r-- 1 appuser appuser 2.6G Jul 15 20:40 _kft.prx -rw-r--r-- 1 appuser appuser 31 Jul 15 20:40 _kft.stats -rw-r--r-- 1 appuser appuser 20K Jul 19 23:01 _kft_sv.del -rw-r--r-- 1 appuser appuser 295K Jul 15 20:40 _kft.tii -rw-r--r-- 1 appuser appuser 25M Jul 15 20:40 _kft.tis -rw-r--r-- 1 appuser appuser 6.6K Jul 19 18:32 _ls6_aj.del -rw-r--r-- 1 appuser appuser 17M Jul 17 18:21 _ls6.fdt -rw-r--r-- 1 appuser appuser 418K Jul 17 18:21 _ls6.fdx -rw-r--r-- 1 appuser appuser 397 Jul 17 18:21 _ls6.fnm -rw-r--r-- 1 appuser appuser 556M Jul 17 19:13 _ls6.frq -rw-r--r-- 1 appuser appuser 1.2M Jul 17 19:13 _ls6.nrm -rw-r--r-- 1 appuser appuser 2.9G Jul 17 19:13 _ls6.prx -rw-r--r-- 1 appuser appuser 31 Jul 17 19:13 _ls6.stats -rw-r--r-- 1 appuser appuser 155K Jul 17 19:13 _ls6.tii -rw-r--r-- 1 appuser appuser 14M Jul 17 19:13 _ls6.tis -rw-r--r-- 1 appuser appuser 20 Jul 27 14:44 segments.gen -rw-r--r-- 1 appuser appuser 158 Jul 27 14:44 segments_pg5 [r...@sc-s1-172-1.oxford.com ~]# [r...@sc-s1-172-1.oxford.com ~]# ls -lh /index/TP_499/ | grep G total 9.7G -rw-r--r-- 1 appuser appuser 2.2G Jul 27 13:58 _12r7.prx -rw-r--r-- 1 appuser appuser 2.6G Jul 15 20:40 _kft.prx -rw-r--r-- 1 appuser appuser 2.9G Jul 17 19:13 _ls6.prx [r...@sc-s1-172-1.oxford.com ~]# Regards Ravi
Indexing and Searching fields that have unique values
Using Lucene.Net I've built an index of documents. The documents also have a unique identifier (my identifier, not the lucene index's id). The unique identifers are also a sort order of new-ness (higher id values are newer) string my_id =1234 doc.Add(new Field(id, my_id, Field.Store.YES, Field.Index.UN_TOKENIZED)); Searching for a particular id, or range searches are incredibly slow TermQuery query = new TermQuery(new Term(id, 1234)); searcher.Search(query) Any tips on how to speed up such an search? I'm also doing RangeSearches on lower / upper ids, and those are slow too _ The New Busy is not the too busy. Combine all your e-mail accounts with Hotmail. http://www.windowslive.com/campaign/thenewbusy?tile=multiaccountocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_4
Re: Version.onOrAfter() complaing its Deprecated but it isnt
I would like to say thanks first. But I have a small problem on both prefix
and Fuzzy search. Is it possible to perform both Prefix and Fuzzy search at
time. I mean to say using FuzzyPrefixLength and setFuzzyMinSim.
Please suggest me ASAP.
On Fri, Mar 19, 2010 at 5:17 PM, Ian Lea ian@gmail.com wrote:
1. Please ask new questions in new threads.
2. Read the Why am I getting no hits / incorrect hits? section of
the FAQ. If that doesn't help post again (in a new thread) showing us
how you are indexing the search field and creating the query and what
the toString() method shows. And use Luke to check that you are
indexing what you think you are.
--
Ian.
On Fri, Mar 19, 2010 at 11:39 AM, vhanuman kumarbaburavi
v.r...@niit-tech.com wrote:
Hi Paul,
I need some help on Lucene prefix search. I have implemented prefix
search in my application but it is return different result like following
way.
E.g. three products see below
Abc def zse,
def sde sed,
fed fer def.
I entered the characters like def and I performed prefix search, I got
following results.
Abc def zse
def sde sed
fed fer def.
But I want only one result instead of above result i.e. def sde sed
Please help me.
Thanks Regards
|Sr.Software Engineer| NIIT Technologies Ltd.
Tapasya(Corporate Heights), Plot No. 5, EFGH, Sector 126,
Greater Noida Express Way, NOIDA(Office: Ex.No 51523 Mb: 9958381449)
| Email:- vhanuma...@nii-tech.com|www.niit-tech.com|
-Original Message-
From: Paul Taylor [mailto:paul_t...@fastmail.fm]
Sent: Friday, March 19, 2010 5:02 PM
To: java-user@lucene.apache.org
Subject: Version.onOrAfter() complaing its Deprecated but it isnt
Hi since downloading Lucene 3.1 my code complains that
Version.onOrAfter() complaing its deprecated but i also have svn access
to the source and it isn't deprecated , and doesnt look like it ever has
been, anyone else get this ?
Paul
-
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
DISCLAIMER
The content of this email and any attachments ('email') is confidential,
may be privileged, subject to copyright and may be read and used only by the
intended recipient. If you are not the intended recipient please notify us
by return email or telephone and erase all copies and do not disclose the
email or any part of it to any person. Email transmission cannot be
guaranteed to be secure, or error free as information could be intercepted,
corrupted, lost or destroyed as a result of the transmission process. The
sender, therefore, does not accept liability for any errors, omissions,
viruses or delay in transmission as a result of this mail. We monitor email
communications through our networks for regulatory compliance purposes and
to protect our clients, employees and business. Opinions, conclusions, and
other information in this message that do not relate to the official
business of NIIT Technologies Ltd. or its affiliate(s) shall be understood
to be neither given nor endorsed by NIIT Technologies Ltd. or its
affiliate(s).
-
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
-
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
--
Thanks
--kumarbabu
Discouragement and failure are two of the surest stepping stones to success.
- Dale Carnegie
Simple Java Object Search
All, I have a list of java objects and would like to index the contents of those objects. And would like to update the index whenever list of objects is changed. The big question is when users searches for something in index, I would like to get all the objects that matached that search string. Would somebody help me in this? Thanks, Ravi Ambati -- This message and any attachments are confidential, proprietary, and may be privileged. If this message was misdirected, Barclays Global Investors (BGI) does not waive any confidentiality or privilege. If you are not the intended recipient, please notify us immediately and destroy the message without disclosing its contents to anyone. Any distribution, use or copying of this e-mail or the information it contains by other than an intended recipient is unauthorized. The views and opinions expressed in this e-mail message are the author's own and may not reflect the views and opinions of BGI, unless the author is authorized by BGI to express such views or opinions on its behalf. All email sent to or from this address is subject to electronic storage and review by BGI. Although BGI operates anti-virus programs, it does not accept responsibility for any damage whatsoever caused by viruses being passed. - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
RE: Simple Java Object Search
Hi,
Getting started shows how to index files. It does not have an example of
how to index how to index a list of objects. I was able to create an
index for the list of objects.
How do I retrieve a list of objects that match the query ?
// Create Index
Directory directory = new RAMDirectory();
Analyzer analyzer = new StandardAnalyzer();
IndexWriter iwriter = null;
try {
iwriter = new IndexWriter(directory, analyzer, true);
iwriter.setMaxFieldLength(25);
EventListTestObject list = getAllBaskets(basketCache);
for (TestObject obj : list) {
Document document = new Document();
document.add(new Field(basketName,
basket.getName(), Field.Store.YES, Field.Index.TOKENIZED));
document
.add(new Field(borrower,
basket.getBorrowerCode(), Field.Store.YES, Field.Index.TOKENIZED));
document.add(new Field(description,
basket.getDescription(), Field.Store.YES,
Field.Index.TOKENIZED));
document.add(new Field(id, new
Long(basket.getId()).toString(), Field.Store.YES,
Field.Index.TOKENIZED));
iwriter.addDocument(document);
}
iwriter.optimize();
iwriter.close();
// Now search the index:
IndexSearcher isearcher = new IndexSearcher(directory);
// Parse a simple query that searches for text:
QueryParser parser = new QueryParser(basketName,
analyzer);
Query query = parser.parse(searchBox.getText());
Hits hits = isearcher.search(query);
// Iterate through the results:
ListLong basketIds = new ArrayListLong();
for (int i = 0; i hits.length(); i++) {
Document hitDoc = hits.doc(i);
}
isearcher.close();
directory.close();
-Original Message-
From: Garth Patil [mailto:garthpa...@gmail.com]
Sent: Thursday, February 26, 2009 9:11 AM
To: java-user@lucene.apache.org
Subject: Re: Simple Java Object Search
Sure:
http://lucene.apache.org/java/2_4_0/gettingstarted.html
On Thu, Feb 26, 2009 at 9:06 AM, Ambati, Ravi BGI SF
ravi.amb...@barclaysglobal.com wrote:
All,
I have a list of java objects and would like to index the contents of
those objects. And would like to update the index whenever list of
objects is changed.
The big question is when users searches for something in index, I
would like to get all the objects that matached that search string.
Would somebody help me in this?
Thanks,
Ravi Ambati
--
This message and any attachments are confidential, proprietary, and
may be privileged. If this message was misdirected, Barclays Global
Investors (BGI) does not waive any confidentiality or privilege. If you
are not the intended recipient, please notify us immediately and destroy
the message without disclosing its contents to anyone. Any distribution,
use or copying of this e-mail or the information it contains by other
than an intended recipient is unauthorized. The views and opinions
expressed in this e-mail message are the author's own and may not
reflect the views and opinions of BGI, unless the author is authorized
by BGI to express such views or opinions on its behalf. All email sent
to or from this address is subject to electronic storage and review by
BGI. Although BGI operates anti-virus programs, it does not accept
responsibility for any damage whatsoever caused by viruses being passed.
-
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
-
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
--
This message and any attachments are confidential, proprietary, and may be
privileged. If this message was misdirected, Barclays Global Investors (BGI)
does not waive any confidentiality or privilege. If you are not the intended
recipient, please notify us immediately and destroy the message without
disclosing its contents to anyone. Any distribution, use or copying of this
e-mail or the information it contains by other than an intended recipient is
unauthorized. The views and opinions expressed in this e-mail message are the
author's own and may not reflect the views and opinions of BGI, unless the
author is authorized by BGI to express such views or opinions on its behalf.
All email sent to or from this address is subject to electronic storage and
review by BGI. Although BGI operates anti-virus programs
Re: Storing part of the field
Thanks Anshum! This can be possible. But, I am searching for is to do this with only one field. thanks ravi On 14-Nov-08, at 1:32 PM, Anshum wrote: Hi Ravi, In that case, you could have 2 fields. One of them would be indexed (i.e. foo bar) and you could use the other only to store as per your logic. Hope this solves your purpose. -- Anshum Gupta Naukri Labs! http://ai-cafe.blogspot.com The facts expressed here belong to everybody, the opinions to me. The distinction is yours to draw On Fri, Nov 14, 2008 at 1:19 PM, Ravi L [EMAIL PROTECTED] wrote: Hi, I am simple question. I want a string to be indexed, but stored part of that string. For example, if my string is foo bar, I want to index whole string(foo bar) but store the first 3 characters(foo) of it. How can I do this with the lucene APIs? thanks ravi - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Storing part of the field
Thanks Erick! The application which uses the index expects this in same field. So, can't use two fields. Any ways, Thank you guys for quick your responses! thanks ravi On 14-Nov-08, at 6:38 PM, Erick Erickson wrote: As far as I know you can't do this with just one field. Why do you care? Storing two fields, one indexed but not stored and one stored but not indexed shouldn't use very many resources. Best Erick On Fri, Nov 14, 2008 at 3:06 AM, Ravi L [EMAIL PROTECTED] wrote: Thanks Anshum! This can be possible. But, I am searching for is to do this with only one field. thanks ravi On 14-Nov-08, at 1:32 PM, Anshum wrote: Hi Ravi, In that case, you could have 2 fields. One of them would be indexed (i.e. foo bar) and you could use the other only to store as per your logic. Hope this solves your purpose. -- Anshum Gupta Naukri Labs! http://ai-cafe.blogspot.com The facts expressed here belong to everybody, the opinions to me. The distinction is yours to draw On Fri, Nov 14, 2008 at 1:19 PM, Ravi L [EMAIL PROTECTED] wrote: Hi, I am simple question. I want a string to be indexed, but stored part of that string. For example, if my string is foo bar, I want to index whole string(foo bar) but store the first 3 characters(foo) of it. How can I do this with the lucene APIs? thanks ravi - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
:using different analyzers for indexing and searching:
Hi , I am using Standard Analyzer with stop words when I am trying to index my documents.. but when I want to support wildcard functionality it is not working with Standard Analyzer since it ignores the Punctuation character where * and ? are also part of Punctuations . So I want to use another analyzer for searcher. If I do like this will I get my required result or not.. Most importantly I would like to know is this a good practice to have two different analyzers for two operations or same analyzer.. Thanks Ravi Kumar Jaladanki
:Lucene 1.9 RC1 is not working properly with older version of Code 1.43:
Hi , I got the latest source code of Lucene 1.9 RC1 and modified my code according to that by removing the deprecated methods. But once I have updated to this version the search is not working at all.. if I try with luke it is working fine but If I try with program it is not returning any error .. and result.. please let me know If any problems are there with this version. If that contains I will replace with old version1.4.3 which is working fine.. Thanks Ravi Kumar Jaladanki
RE: Suggesting refine searches with Lucene
Hi ,
I have implemented by using query mergeBooleanQueries method... in this
approach I have created one POJO class of RefineQuery which contains one
variable called Query and I will set whenever I get a search..
And next time if it is a refined search I will merge current query with the
refinedquery object and get new query and pass to lucene and set the new
query to refined search object this is working fine.. let me know if
have any further ideas or any new technique to implement refined search
if(objSearchParameters.isBSearchInSearch()){
Query q2=
Query.mergeBooleanQueries(new Query[]{ objRefineQuery.getQuery(),
booleanQuery });
objRefineQuery.setQuery(q2);
hits=searcher.search(q2);
}else{
objRefineQuery.setQuery(booleanQuery);
hits= searcher.search(booleanQuery);
}
public class RefineQuery {
private Query query = null;
public Query getQuery() {
return query;
}
public void setQuery(Query query) {
this.query = query;
}
public String toString(){
return query.toString();
}
}
Regards,
Ravi Kumar Jaladanki
-Original Message-
From: Chun Wei Ho [mailto:[EMAIL PROTECTED]
Sent: Monday, February 13, 2006 3:05 PM
To: java-user@lucene.apache.org
Subject: Suggesting refine searches with Lucene
Hi,
I am trying to suggest refine searches for my Lucene search. For
example, if a search turned out too many searches, it would list a
number of document title subsequences that occurred frequently in the
results of the previous search, as possible candidates for refining
the search.
Does anyone know the right/any approach to implementing this in a
Lucene-based search app?
Thanks.
CW
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Memory Increasing when the optimize is called
Hi
I have index file around 2GB and when I optimize the index file tomcat is
taking more memory than normal and even after completion of optimization
also it is still taking more memory than the normal memory.. is this way it
will happen or do I need to change any thing to reduce the memory I am
sending the code for optimization .. Please tell me is there any wrong in
the following code.
public static void optimizeIndexing() {
System.out.println(Index Optitmaization
started);
ERMProperties objProperties
= new ERMProperties();
String strApplicationDataPath
= objProperties.getProperty(applicationDataPath);
String strIndexPath
= strApplicationDataPath + Index;
File objIndexDir
= new File(strIndexPath);
boolean bNewIndex
= false;
IndexWriter indexWriter
= null;
FSDirectory objFSDirectory
= null;
try{
if
(!objIndexDir.isDirectory()) {
objIndexDir.mkdir();
bNewIndex
= true;
}
objFSDirectory
= FSDirectory.getDirectory(new File(strIndexPath), bNewIndex);
indexWriter
= new IndexWriter(objFSDirectory, new HLAnalyzer(), bNewIndex);
indexWriter.optimize();
}catch(Exception objException){
objException.printStackTrace();
}finally{
try { if (indexWriter !=
null) indexWriter.close(); } catch( Exception exc) { }
try { if (objFSDirectory !=
null) objFSDirectory.close(); } catch( Exception exc) { }
}
System.out.println(Index Optitmaization
Endeded);
}
Regards,
Ravi Kumar Jaladanki
RE: :intersection of two hits objects:
Thanks for your valuable suggestions to me.. I am very much glad to you for this response. Now I understood where I am going wrong so I will try use the first solution given by you Thanks Ravi Kumar Jaladanki -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Chris Hostetter Sent: Thursday, January 19, 2006 12:51 PM To: java-user@lucene.apache.org Subject: RE: :intersection of two hits objects: The first thing you need to keep in mind, is that a Hits object doesn't contain results. The Hits class is provided as a convincent way to access results in order, that provides some prefecthing of ids/scores, and caching of doc stored fields. Iterating over an entire Hits object is *never* a good idea. Depending on how big your result set is, iterating over every doc in the results with a Hits object will re-execute your search over and over again as you reach the end of it's buffer. If it's not entirely obvious yet: there is no way to make a Hits object and add or remove documents to/from it. all a Hits objects has in it is your orriginal Query, a refrnece to your searcher, and a cache. if you need a Hits obejct ot contain only the results that can be found in both of two queries A and B, then you should do one of two things: 1) combine A and B in a boolean query ... this is the right option if you want the scores in your results to be based on the aggregate scores from A and B. 2) execute A filtered by B (or B filtered by A) ... this is the right option if you want the scores in your results to be based on just one query ... and the other should only be used to restrict the results. if you don't have the orriginal queries A and B, and all you have is the Hits that resulted from those queries ... then i suggest you change your app. : Date: Thu, 19 Jan 2006 11:42:41 +0530 : From: Ravi [EMAIL PROTECTED] : Reply-To: java-user@lucene.apache.org : To: java-user@lucene.apache.org : Subject: RE: :intersection of two hits objects: : : Thanks for your help , : : I am able to get the record numbers in BitSet object but how to iterate in : the first hits1 object with the bitset object and retain only those : documents which contains ids existed in bitset object.. : : : Thanks : Ravi Kumar jaladanki : : -Original Message- : From: Dave Kor [mailto:[EMAIL PROTECTED] : Sent: Thursday, January 19, 2006 7:01 AM : To: java-user@lucene.apache.org : Subject: Re: :intersection of two hits objects: : : On 1/18/06, Ravi [EMAIL PROTECTED] wrote: : Hi : : I want to find out the intersection of two hits objects please help me : : Hits hits1 = Searcher.searh(strQuery1); : Hits hits2 = Searcher.searh(strQuery2); : : Now In hits1 contains records numbers 3,4 ,5,6 and : Hits2 contains records numbers 3,6,8,9 : : Now I need a solution which can give the hits object which contains 3,6 : records : : : You can iterate through the Hits objects, flagging the document : numbers in a java.util.BitSet. To compare hits between different : queries, all you have to do is bitset1.and(bitset2). : : -- : Dave Kor, Research Assistant : Center for Information Mining and Extraction : School of Computing : National University of Singapore. : : - : To unsubscribe, e-mail: [EMAIL PROTECTED] : For additional commands, e-mail: [EMAIL PROTECTED] : : : - : To unsubscribe, e-mail: [EMAIL PROTECTED] : For additional commands, e-mail: [EMAIL PROTECTED] : -Hoss - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
:Highlighting the Specific Filed:
Hi, I want to highlight the specific field which I have added to document. For ex I have added two fileds one is Contens and second one is Skills So I want highlight Skills which I have added as Filed.Keyword(Skills,Java, oracle) Please help me I have tried out the example in the lucene action book . but I didn't get any solution .Can any body gives the code for that to highlight specific field from document Thanks Ravi Kumar Jaladanki
RE: :intersection of two hits objects:
Thanks But I will get these Hits objects in two cases. First I will search and store it in temp location and next time I search I get the new hits object . now I need to get the intersection Thanks Ravi Kumar Jaladanki -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Wednesday, January 18, 2006 7:13 PM To: java-user@lucene.apache.org Cc: Ravi Subject: Re: :intersection of two hits objects: Why not combine the two searches with an AND operator (search 1) AND (search2) Regards Marco Ravi [EMAIL PROTECTED] schrijft: Hi I want to find out the intersection of two hits objects please help me Hits hits1 = Searcher.searh(strQuery1); Hits hits2 = Searcher.searh(strQuery2); Now In hits1 contains records numbers 3,4 ,5,6 and Hits2 contains records numbers 3,6,8,9 Now I need a solution which can give the hits object which contains 3,6 records I would appreciate any help from all Thanks Ravi Kumar Jaladanki - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: :intersection of two hits objects:
Thanks for your help , I am able to get the record numbers in BitSet object but how to iterate in the first hits1 object with the bitset object and retain only those documents which contains ids existed in bitset object.. Thanks Ravi Kumar jaladanki -Original Message- From: Dave Kor [mailto:[EMAIL PROTECTED] Sent: Thursday, January 19, 2006 7:01 AM To: java-user@lucene.apache.org Subject: Re: :intersection of two hits objects: On 1/18/06, Ravi [EMAIL PROTECTED] wrote: Hi I want to find out the intersection of two hits objects please help me Hits hits1 = Searcher.searh(strQuery1); Hits hits2 = Searcher.searh(strQuery2); Now In hits1 contains records numbers 3,4 ,5,6 and Hits2 contains records numbers 3,6,8,9 Now I need a solution which can give the hits object which contains 3,6 records You can iterate through the Hits objects, flagging the document numbers in a java.util.BitSet. To compare hits between different queries, all you have to do is bitset1.and(bitset2). -- Dave Kor, Research Assistant Center for Information Mining and Extraction School of Computing National University of Singapore. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: :intersection of two hits objects:
Hi Erik Can u please tell me how to use this query in loop because he can refine the search n number of time so how to maintain all the queries in QueryFilter and use of them , Please help me I need very urgent. Thanks Ravi Kumar Jaladanki -Original Message- From: Erik Hatcher [mailto:[EMAIL PROTECTED] Sent: Wednesday, January 18, 2006 7:12 PM To: java-user@lucene.apache.org Subject: Re: :intersection of two hits objects: On Jan 18, 2006, at 8:20 AM, Ravi wrote: I want to find out the intersection of two hits objects please help me Hits hits1 = Searcher.searh(strQuery1); Hits hits2 = Searcher.searh(strQuery2); Now In hits1 contains records numbers 3,4 ,5,6 and Hits2 contains records numbers 3,6,8,9 Now I need a solution which can give the hits object which contains 3,6 records One option is to construct a QueryFilter with the first query, and use the search(Query, Filter) method. Another option is to AND the two queries together by wrapping them in a BooleanQuery with both clauses required. Erik - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Retriving the data from lucene
Thanks Erik, I have already have the copy of the book and I am referencing the book to solve the problems .You have given very nice examples to solve the problems. I am going on reading this book to solve my problems . But I still have following problems . 1) I want to store all the details of the my data in the lucene and I want to retrieve from the lucene than storing in the database and retrieve that from database .Is this technique is good or bad or any other solution to this type of problem Please suggest me. 2) I have one more problem is I want to retrieve the max 1000 docs from the search. How can I solve this problem . 3) I want to display page by page max 100. a) load all the data at a time and store in temp location and display 1-100 ,101-200 and so on 900-1000. b) load only first 1-00 and whenever user clicks next 100 I want to display next hundred by loading or searching again.. I am not able to figure out how to get the details like this.. Could you please help to solve this problem . Please I need to submit to my Seniors on this weekend... I want to have really good solution which can not take much time in terms of performance... I am waiting for your kind reply Thanks Ravi -Original Message- From: Erik Hatcher [mailto:[EMAIL PROTECTED] Sent: Tuesday, December 13, 2005 7:28 PM To: java-user@lucene.apache.org Subject: Re: :how to add int fileds to lucene: Ravi, A few great starting points... the code from http:// www.lucenebook.com (and it wouldn't hurt my feelings if you picked up a copy of the book itself too :), the examples from the many articles that have been written on Lucene, and last but not least, the unit tests of Lucene itself which is a great place to learn the API. Erik On Dec 12, 2005, at 10:32 PM, Ravi wrote: Hi Erik Thanks for your solution. I want to do exactly what you have mentioned in the mail . I would like to search on the fields what I have added to lucene .That is search on days and sort on those days value and moreover how to add multiple field querys to lucene . I don't have any idea how to Query on multiple fields .ex searching on the candidate and days where days 5 and candidate10 or days one month and so on.. Please give me the code or let me know where I can find test programs so that I can complete my project. Waiting for your reply Thanks Ravi Kumar Jaladanki 408-328-6437 -Original Message- From: Erik Hatcher [mailto:[EMAIL PROTECTED] Sent: Monday, December 12, 2005 9:35 PM To: java-user@lucene.apache.org Subject: Re: :how to add int fileds to lucene: On Dec 12, 2005, at 8:10 AM, Ravi wrote: I am trying to add some fields to lucene and I heard that adding int values are going to give much faster retrieval than adding to String values. So I want to add int values to document . But document.add(Field.Text(Candidate, objResultSet.getString (ROW_ID))); document.add(Field.Keyword(lastmodified, objResultSet.getDate(MODIFIED_ON))); document.add(Field.Text(days,days) ); document.add(Field.Text(contents, objReader)); Days are int but it is giving error . Please suggest me with the steps to add and search those integer fields All stored field data in a Lucene index are Strings. The performance difference you may be referring to is with sorting results, not in retrieval. You will want to simply convert your int days into a String (Integer.toString(days) will do the trick. Careful with your analyzer if you really do want days to be searchable/sortable. Using Field.Keyword() is probably more appropriate. Erik - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
:Creating the search on last modified list value:
Hi , I want create a following search query search and Return results updated in theany time Last 7 days Last 2 weeks Last 1 Month so on. http://svn.apache.org/repos/asf/lucene/java/trunk/contrib/javascript/queryCo nstructor/luceneQueryConstructor.html This is just like what they have mentioned in sand box. Of Lucene as above link . but I don't know how to construct this query and search in the index.. Pls help me with the source code or link which can give me the details to construct and search the data based on above condition. And moreover I want to return only top 1000 and want to display them just like hot jobs or dice how they are showing in the search results.. Please help me by giving the suitable solution to this problem
:how to add int fileds to lucene:
Hi, I am trying to add some fields to lucene and I heard that adding int values are going to give much faster retrieval than adding to String values. So I want to add int values to document . But document.add(Field.Text(Candidate, objResultSet.getString(ROW_ID))); document.add(Field.Keyword(lastmodified, objResultSet.getDate(MODIFIED_ON))); document.add(Field.Text(days,days) ); document.add(Field.Text(contents, objReader)); Days are int but it is giving error . Please suggest me with the steps to add and search those integer fields Thanks Ravi Kumar Jaladanki
RE: :how to add int fileds to lucene:
Hi Erik Thanks for your solution. I want to do exactly what you have mentioned in the mail . I would like to search on the fields what I have added to lucene .That is search on days and sort on those days value and moreover how to add multiple field querys to lucene . I don't have any idea how to Query on multiple fields .ex searching on the candidate and days where days 5 and candidate10 or days one month and so on.. Please give me the code or let me know where I can find test programs so that I can complete my project. Waiting for your reply Thanks Ravi Kumar Jaladanki 408-328-6437 -Original Message- From: Erik Hatcher [mailto:[EMAIL PROTECTED] Sent: Monday, December 12, 2005 9:35 PM To: java-user@lucene.apache.org Subject: Re: :how to add int fileds to lucene: On Dec 12, 2005, at 8:10 AM, Ravi wrote: I am trying to add some fields to lucene and I heard that adding int values are going to give much faster retrieval than adding to String values. So I want to add int values to document . But document.add(Field.Text(Candidate, objResultSet.getString (ROW_ID))); document.add(Field.Keyword(lastmodified, objResultSet.getDate(MODIFIED_ON))); document.add(Field.Text(days,days) ); document.add(Field.Text(contents, objReader)); Days are int but it is giving error . Please suggest me with the steps to add and search those integer fields All stored field data in a Lucene index are Strings. The performance difference you may be referring to is with sorting results, not in retrieval. You will want to simply convert your int days into a String (Integer.toString(days) will do the trick. Careful with your analyzer if you really do want days to be searchable/sortable. Using Field.Keyword() is probably more appropriate. Erik - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
FileNotFound Exception
I'm getting the following exception when I try to update my searcher object. When I checked the index folder it does not have the .fnm file. I have .cfs files, a deletable and a segments file. Has anybody had similar problem? Thanks in advance, Ravi. The class java.io.FileNotFoundException's message follows:/a/newsindex/20050502173800/_6i2f.fnm (No such file or directory)The Stack trace follows:java.io.FileNotFoundException: /a/index/20050502173800/_6i2f.fnm (No such file or directory) at java.io.RandomAccessFile.open(Native Method) at java.io.RandomAccessFile.init(RandomAccessFile.java(Compiled Code)) at org.apache.lucene.store.FSInputStream$Descriptor.init(FSDirectory.java (Inlined Compiled Code)) at org.apache.lucene.store.FSInputStream.init(FSDirectory.java(Inlined Compiled Code)) at org.apache.lucene.store.FSDirectory.openFile(FSDirectory.java(Compiled Code)) at org.apache.lucene.index.FieldInfos.init(FieldInfos.java(Compiled Code)) at org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java(Comp iled Code)) at org.apache.lucene.index.SegmentReader.init(SegmentReader.java(Inlined Compiled Code)) at org.apache.lucene.index.IndexReader$1.doBody(IndexReader.java(Compiled Code)) at org.apache.lucene.store.Lock$With.run(Lock.java(Compiled Code)) at org.apache.lucene.index.IndexReader.open(IndexReader.java(Inlined Compiled Code)) at org.apache.lucene.index.IndexReader.open(IndexReader.java(Inlined Compiled Code)) at org.apache.lucene.search.IndexSearcher.init(IndexSearcher.java(Inlined Compiled Code)) - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: how to detect index integrity?
From: [EMAIL PROTECTED] Sent: Fri 3/18/2005 11:34 PM Is there any way to detect the index's integrity? Sometimes I came upon exceptions like these. If it happens, my only way is to delete the corrupted index. * Exception in thread main java.io.IOException : read past EOF * java.lang.ArrayIndexOutOfBoundsException [ ... ] I did too, which is why I wrote NullDirectory. You can find the sources and a description in bugzilla. http://issues.apache.org/bugzilla/show_bug.cgi?id=33851 Look at the tests for examples of use. I would value your feedback. -- Ravi/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
