Il giorno mer 26 feb 2020 alle ore 19:30 Dario Rigolin <
dario.rigo...@comperio.it> ha scritto:
> You can avoid commit and leave solr do autocommit at certain times.
> Or use softcommit if you have search queries at the same time to answer.
> 550000 pages of 3500 words isn't a big deal for a solr server, what's the
> hardware configuration?
>
The solr instance runs on a server with the following configuration:
12 core Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
64GB Ram
solr's DataDir is on a volume of another server that I mounted via NFS (I
was thinking of moving the solr server to the server where the DataDir
resides even if it has lower characteristics 8 core Intel(R) Xeon(R) CPU
E5506 @ 2.13GHz 24GB Ram)
What's you single solr document a single newspaper? a single page?
>
the single solr document refers to the single word of the document
> Do you have a solrcloud with 8 nodes? Or are you sending same document to 8
> single solr servers?
>
> I have 8 servers that process 550,000 newspapers and all of them write on
1 solr server only
> Il giorno mer 26 feb 2020 alle ore 19:22 Massimiliano Randazzo <
> massimiliano.randa...@gmail.com> ha scritto:
>
> > Good morning
> >
> > I have the following situation I have to index the OCR of about 550,000
> > pages of newspapers counting an average of 3,500 words per page and
> making
> > a document per word the records are many.
> >
> > At the moment I have 1 instance of Solr and 8 servers that read and write
> > all on the same instance at the same time, at the beginning everything is
> > fine after a while when I add, delete or commit it gives me a TimeOut
> error
> > towards the solr server.
> >
> > I suspect the problem is due to the fact that it is that I do many commit
> > operations of many docs at a time (practically if the newspaper is 30
> pages
> > I do 105,000 add and in the end I commit), if everyone does this and 8
> > servers within walking distance of each other I think this creates
> problems
> > for Solr.
> >
> > What can I do to solve the problem?
> > Do I make a commi to each add?
> > Is it possible to configure the solr server to apply the add and delete
> > commands, and to commit it, the server autonomously supports the
> available
> > resources as it seems to do for the optmized command?
> > Reading the documentation I would have found this configuration to
> > implement but not if it solves my problem
> >
> > <deletionPolicy class="solr.SolrDeletionPolicy">
> > <str name="maxCommitsToKeep">1</str>
> > <str name="maxOptimizedCommitsToKeep">0</str>
> > <str
> >
> name="maxCommitAge">1DAY</str></deletionPolicy><infoStream>false</infoStream>
> >
> >
> >
> > Thanks for your consideration
> > Massimiliano Randazzo
> >
>
>
> --
>
> Dario Rigolin
> Comperio srl - CTO
> Mobile: +39 347 7232652 - Office: +39 0425 471482
> Skype: dario.rigolin
>
--
Massimiliano Randazzo
Analista Programmatore,
Sistemista Senior
Mobile +39 335 6488039
email: massimiliano.randa...@gmail.com
pec: massimiliano.randa...@pec.net
