Juhu, great news, guys. I merged my child entity into the root entity,
and changed the custom entityprocessor to handle the additional columns
correctly.
And - indexing 160k documents now takes 5min instead of 1.5h!
(Now I can go relaxed on vacation. :-D )
Conclusion:
In my case performance was so bad because of constantly querying a
database on a different machine (network traffic + db query per document).
Thanks for all your help!
Chantal
Avlesh Singh schrieb:
does DIH call commit periodically, or are things done in one big batch?
AFAIK, one big batch.
yes. There is no index available once the full-import started (and the
searcher has no cache, other wise it still reads from that). There is no
data (i.e. in the Admin/Luke frontend) visible until the import is
finished correctly.
