Hello
Firstly great work on every one involved in the Lucene.NET its a great
project. We are going to use it as the search for a new project that we
are working on which is due to go live in the next few months. The only
problem is that we are getting great results from the search if we are
only searching one field but generally our users will be selecting from
two or three fields which using the MultiFieldQueryParser does not seem
to be producing great results so I was wondering if any one might be
able to help us?
To create the index I am using the code below which is written to disk:-
luceneDocument.Add(new
Field(Constant.LuceneConstant_FoodItemId,
foodItem.FoodItemId.ToString(), Field.Store.YES,
Field.Index.UN_TOKENIZED, Field.TermVector.NO));
luceneDocument.Add(new
Field(Constant.LuceneConstant_ParentFoodItemId,
foodItem.ParentFoodItemId.ToString(), Field.Store.YES,
Field.Index.UN_TOKENIZED, Field.TermVector.NO));
luceneDocument.Add(new
Field(Constant.LuceneConstant_DataProviderItemId,
foodItem.DataProviderItemId.ToString(), Field.Store.YES,
Field.Index.UN_TOKENIZED, Field.TermVector.NO));
luceneDocument.Add(new
Field(Constant.LuceneConstant_EANcode,
(foodItem.EuropeanArticleNumberCode == null) ? string.Empty :
foodItem.EuropeanArticleNumberCode.ToString(), Field.Store.YES,
Field.Index.UN_TOKENIZED, Field.TermVector.NO));
luceneDocument.Add(new
Field(Constant.LuceneConstant_Summary, foodItem.Summary,
Field.Store.YES, Field.Index.TOKENIZED, Field.TermVector.NO));
luceneDocument.Add(new
Field(Constant.LuceneConstant_StorageTypeId,
foodItem.StorageTypeId.ToString(), Field.Store.YES,
Field.Index.UN_TOKENIZED, Field.TermVector.NO));
luceneDocument.Add(new
Field(Constant.LuceneConstant_FoodItemSupplierId,
foodItem.FoodItemSupplierId.ToString(), Field.Store.YES,
Field.Index.UN_TOKENIZED, Field.TermVector.NO));
luceneDocument.Add(new
Field(Constant.LuceneConstant_FoodItemBrandId,
foodItem.FoodItemBrandId.ToString(), Field.Store.YES,
Field.Index.UN_TOKENIZED, Field.TermVector.NO));
luceneDocument.Add(new
Field(Constant.LuceneConstant_MeasurementType,
((int)foodItem.MeasurementType).ToString(), Field.Store.YES,
Field.Index.UN_TOKENIZED, Field.TermVector.NO));
luceneDocument.Add(new
Field(Constant.LuceneConstant_NoOfUnits, foodItem.NoOfUnits.ToString(),
Field.Store.YES, Field.Index.UN_TOKENIZED, Field.TermVector.NO));
luceneDocument.Add(new
Field(Constant.LuceneConstant_PackSize, foodItem.PackSize.ToString(),
Field.Store.YES, Field.Index.UN_TOKENIZED, Field.TermVector.NO));
luceneDocument.Add(new
Field(Constant.LuceneConstant_PortionSize,
foodItem.PortionSize.ToString(), Field.Store.YES,
Field.Index.UN_TOKENIZED, Field.TermVector.NO));
luceneDocument.Add(new
Field(Constant.LuceneConstant_Approved, foodItem.Approved.ToString(),
Field.Store.YES, Field.Index.UN_TOKENIZED, Field.TermVector.NO));
luceneDocument.Add(new
Field(Constant.LuceneConstant_Created, foodItem.Created.ToString(),
Field.Store.YES, Field.Index.UN_TOKENIZED, Field.TermVector.NO));
luceneDocument.Add(new
Field(Constant.LuceneConstant_Updated, foodItem.Updated.ToString(),
Field.Store.YES, Field.Index.UN_TOKENIZED, Field.TermVector.NO));
The index contains about 20,000 food products which creates an index of
about 6mb. The code to search this index is then below
public static SortableResultsCollection<FoodItem>
FoodItemSearch(short departmentCode, short ailseCode, short shelfCode,
short supermarketId, byte searchType, string searchQuery, int
startValue, int amountOfResults)
{
List<string> queryFieldList = new List<string>();
List<string> queryList = new List<string>();
List<BooleanClause.Occur> queryClauseList = new
List<BooleanClause.Occur>();
//Process the department code
if (departmentCode > 0)
{
queryList.Add(Constant.LuceneConstant_CategoryTopLevel);
queryFieldList.Add(departmentCode.ToString());
queryClauseList.Add(BooleanClause.Occur.MUST);
}
//Process the ailse code
if (ailseCode > 0)
{
queryList.Add(Constant.LuceneConstant_CategorySecondLevel);
queryFieldList.Add(ailseCode.ToString());
queryClauseList.Add(BooleanClause.Occur.MUST);
}
//Process the shelf code
if (shelfCode > 0)
{
queryList.Add(Constant.LuceneConstant_CategoryThirdLevel);
queryFieldList.Add(shelfCode.ToString());
queryClauseList.Add(BooleanClause.Occur.MUST);
}
//Process the supermarket
if (supermarketId > 0)
{
queryList.Add(Constant.LuceneConstant_FoodItemSupplierId);
queryFieldList.Add(supermarketId.ToString());
queryClauseList.Add(BooleanClause.Occur.MUST);
}
//Process the search query
if (searchQuery != string.Empty)
{
if (searchType == 1)
{
queryList.Add(Constant.LuceneConstant_Summary);
queryFieldList.Add(searchQuery);
queryClauseList.Add(BooleanClause.Occur.MUST);
}
else
{
queryList.Add(Constant.LuceneConstant_EANcode);
queryFieldList.Add(searchQuery);
queryClauseList.Add(BooleanClause.Occur.MUST);
}
}
//Create the arrays to pass to the query
string[] queryFieldArray = new string[queryFieldList.Count];
string[] queryArray = new string[queryList.Count];
BooleanClause.Occur[] occurArray = new
BooleanClause.Occur[queryList.Count];
//Assign the list data to the array
int rowCount = queryList.Count - 1;
for (int i = 0; i <= rowCount; i++)
{
queryFieldArray[i] = queryFieldList[i];
queryArray[i] = queryList[i];
occurArray[i] = queryClauseList[i];
}
Query query = MultiFieldQueryParser.Parse(queryFieldArray,
queryArray, occurArray, new StandardAnalyzer());
EGC.SortableResultsCollection<FoodItem> collection = new
EGC.SortableResultsCollection<FoodItem>();
string indexPath =
ConfigurationManager.AppSettings[CONST_LUCENE_PATH];
IndexSearcher indexSearcher = new IndexSearcher(indexPath);
Hits searchHits = indexSearcher.Search(query);
int totalHitsCount = searchHits.Length();
collection.TotalResultCount = totalHitsCount - 1;
collection.StartResult = startValue;
int hitsCount = startValue + amountOfResults;
if (hitsCount > totalHitsCount)
{
hitsCount = amountOfResults;
}
if (totalHitsCount > 0)
{
for (int hitNumer = startValue; hitNumer <= hitsCount;
hitNumer++)
{
collection.Add(LuceneDocumentToFoodItem(searchHits.Doc(hitNumer)));
}
}
//close the index sercher
indexSearcher.Close();
I know the code at the moment is not amazing but we are trying to get
the concept correct before we do any tuning. I have tried changing
BooleanClause.Occur.MUST to BooleanClause.Occur.SHOULD but we have seen
no improvement in the search. Is using the MultiFieldQueryParser the
correct way to search for products in the index or do I need to do
singular searches and then apply filters to narrow the results?
Any help is greatly received.
Thanks
Ollie Castle
***************************************************************************
E-mail Disclaimer
The information transmitted in this e-mail is intended only for the
confidential use of the named recipient and may contain confidential and/or
privileged material.
Any review, retransmission, dissemination or other use of, or taking of any
action in reliance upon, this information by persons or entities other than the
intended recipient is prohibited.
Furthermore, no part of the information may be reproduced or transmitted in any
form or by any means, electronic or mechanical, or by an information storage or
retrieval system, without prior permission.
TescoDiets may monitor email traffic data and also the content of email for the purposes of security and staff training.
If you received this in error, please contact the sender and/or system manager and delete the material from all relevant computers.
The employer assumes no responsibility for any use to which the information may
be put, or for any errors.
Registration Number 19542
***************************************************************************
