On 4/13/2018 11:03 AM, Jesus Olivan wrote:
> thanks for your answer. It happens that when we launch full import process
> didn't finished (we wait for more than 60 hours last time, and we cancelled
> it, because this is not an acceptable time for us) There weren't any errors
> in solr logfile simply because it was working fine. The problem is that it
> lasted eternally and didn't finish. We tried it on Aurora cluster under
> AWS, and after 20 hours of work, it failed due to lack of space in Aurora
> tmp folder.

375 million documents importing from MySQL with one DIH import is going
to take quite a while.

The last full rebuild I did of my main index took 21.61 hours.  This is
an index where six large shards build simultaneously, using DIH, each
one having more than 30 million documents.  If I were to build it as a
single 180 million document import, it would probably take 5 days, maybe

We had another index (since retired) that had more than 400 million
total documents, built similarly with multiple shards at the same time. 
The last rebuild I can remember on that index took about two days.


Reply via email to