it's a macro:
#define QSTRING_TO_TCHAR(s) (const wchar_t*) s.utf16()
it's up to qt to make a clean up(maybe)
2011/1/9 Ben van Klinken <bvanklin...@gmail.com>
> oh it's a qt wrapper? i have no idea how that works, sorry.
>
> just an idea, but where do you clean up the result from QSTRING_TO_TCHAR?
>
> ben
>
> On Fri, Jan 7, 2011 at 5:57 PM, Ahmed Saidi <ci7nu...@gmail.com> wrote:
> > Hello Muhammad,
> > What analyzer are you using?
> > It seem to me that your code is based on that application
> > "clucene_qt", well i think that you are an old version of that
> > application, it has several bugs concerning this problem, and the
> > ArabicAnalyzer is not well implemented nor optimized, you should use
> > the 0.6 or 0.8 version of this application.
> >
> > Ahmed
> >
> > 2011/1/6, muhammad ismael <m.ismae...@gmail.com>:
> >> Hello ben,
> >> the files are more than 6 GB.
> >> I am sorry i was remember to add this function but i forgot and here it
> is
> >>
> >> Document* IndexEngine::fileDocument(const QString &id, const QString
> &bookid
> >> , const QString &text)
> >>
> >> {
> >>
> >> // make a new, empty document
> >>
> >> Document* doc = _CLNEW Document();
> >>
> >> ///page ID
> >>
> >> doc->add( *_CLNEW Field(_T("id"), QSTRING_TO_TCHAR(id) ,
> >>
> >> Field::STORE_YES | Field::INDEX_UNTOKENIZED)
> );
> >>
> >> doc->add( *_CLNEW Field(_T("bookid"), QSTRING_TO_TCHAR(bookid) ,
> >>
> >> Field::STORE_YES | Field::INDEX_UNTOKENIZED
> ) );
> >>
> >> doc->add( *_CLNEW Field(_T("text"), QSTRING_TO_TCHAR(text),
> >>
> >> Field::STORE_NO | Field::INDEX_TOKENIZED) );
> >>
> >> return doc;
> >>
> >> }
> >>
> >> and also i tried to remove adding documents and the size did not
> increased,
> >> which means that leaks are in addDocument() I am trying to debug it but
> i am
> >> lost.
> >>
> >> sounds pretty high. how big are the files? could you be leaking memory
> in
> >>> the 'fileDocument' function?
> >>>
> >>> as a test, try not actually adding the document
> >>>
> >>> ben
> >>>
> >>> On Thu, Jan 6, 2011 at 7:43 AM, muhammad ismael <m.ismae...@gmail.com
> >>> >wrote:
> >>>
> >>> > Hello,
> >>> > I am trying to index large files as follows
> >>> >
> >>> > for(int j = 0; (j < pagesIds.count())&& !m_stop ; j++)
> >>> >
> >>> > {
> >>> >
> >>> > pagesText =
> >>> m_DbManager->getBookPage(m_booksIds.at(i), pagesIds.at(j)).toUtf8();
> >>> >
> >>> > if(!pagesText.isEmpty())
> >>> >
> >>> > {
> >>> >
> >>> > Document* doc =
> >>> fileDocument(QString::number(pagesIds.at(j)),
> >>> QString::number(m_booksIds.at(i)), pagesText);
> >>> >
> >>> > writer->addDocument(doc);
> >>> >
> >>> > _CLDELETE(doc);
> >>> >
> >>> > }
> >>> >
> >>> > }
> >>> >
> >>> >
> >>> > but when the number of files exceeds 5000 files the application usage
> of
> >>> my
> >>> > computer ram is 2 GB
> >>> > I tried to debug and i found that this happens in
> >>> >
> >>> > IndexWriter::addDocument(Document*)
> >>> >
> >>> > i tried to set
> >>> > IndexWriter->setMergeFactor(5);
> >>> > and also
> >>> > IndexWriter->setRAMBufferSizeMB(10);
> >>> > I know the default ram usage should be 16 MB but i just tried.
> >>> >
> >>> > I am working on master branch and i merged with it memory_leaks
> branch.
> >>> > is am i missing something?
> >>>
> >>
> >> Mohammad Ismael
> >>
> >
> > --
> > Envoyé avec mon mobile
> >
> >
> ------------------------------------------------------------------------------
> > Gaining the trust of online customers is vital for the success of any
> company
> > that requires sensitive data to be transmitted over the Web. Learn how
> to
> > best implement a security strategy that keeps consumers' information
> secure
> > and instills the confidence they need to proceed with transactions.
> > http://p.sf.net/sfu/oracle-sfdevnl
> > _______________________________________________
> > CLucene-developers mailing list
> > CLucene-developers@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/clucene-developers
> >
>
>
> ------------------------------------------------------------------------------
> Gaining the trust of online customers is vital for the success of any
> company
> that requires sensitive data to be transmitted over the Web. Learn how to
> best implement a security strategy that keeps consumers' information secure
> and instills the confidence they need to proceed with transactions.
> http://p.sf.net/sfu/oracle-sfdevnl
> _______________________________________________
> CLucene-developers mailing list
> CLucene-developers@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>
------------------------------------------------------------------------------
Gaining the trust of online customers is vital for the success of any company
that requires sensitive data to be transmitted over the Web. Learn how to
best implement a security strategy that keeps consumers' information secure
and instills the confidence they need to proceed with transactions.
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
CLucene-developers mailing list
CLucene-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/clucene-developers
