Alexandre, did you mean SolrJ? Here's a way to get started https://lucidworks.com/blog/2012/02/14/indexing-with-solrj/
Best, Erick On Sat, Nov 7, 2015 at 2:22 PM, Alexandre Rafalovitch <arafa...@gmail.com> wrote: > Have you thought of just using Solr. Might be faster than troubleshooting > DIH for complex scenarios. > On 7 Nov 2015 3:39 pm, "Yangrui Guo" <guoyang...@gmail.com> wrote: > >> I found multiple strange things besides the slowness. I performed count(*) >> in MySQL but only one-fifth of the records were imported. Also sometimes >> dataimporthandler either doesn't import at all or only imports a portion >> of the table. How can I debug the importer? >> >> On Saturday, November 7, 2015, Yangrui Guo <guoyang...@gmail.com> wrote: >> >> > I just realized that not everything was ok. Three child entities were not >> > imported. Had set batchSize to -1 but again solr was stuck :( >> > >> > On Fri, Nov 6, 2015 at 3:11 PM, Yangrui Guo <guoyang...@gmail.com >> > <javascript:_e(%7B%7D,'cvml','guoyang...@gmail.com');>> wrote: >> > >> >> Thanks for the reply. I just removed CacheKeyLookUp and CachedKey and >> >> used WHERE clause instead. Everything works fine now. >> >> >> >> Yangrui >> >> >> >> >> >> On Friday, November 6, 2015, Shawn Heisey <apa...@elyograg.org >> >> <javascript:_e(%7B%7D,'cvml','apa...@elyograg.org');>> wrote: >> >> >> >>> On 11/6/2015 10:32 AM, Yangrui Guo wrote: >> >>> > <entity name="movie_actress" transformer="RegexTransformer" >> >>> >> >>> There's a good chance that JDBC is trying to read the entire result set >> >>> (all three million rows) into memory before sending any of that info to >> >>> Solr. >> >>> >> >>> Set the batchSize to -1 for MySQL so that it will stream results to >> Solr >> >>> as soon as they are available, and not wait for all of them. Here's >> >>> more info on the situation, which frequently causes OutOfMemory >> problems >> >>> for users: >> >>> >> >>> >> >>> >> http://wiki.apache.org/solr/DataImportHandlerFaq?highlight=%28mysql%29|%28batchsize%29#I.27m_using_DataImportHandler_with_a_MySQL_database._My_table_is_huge_and_DataImportHandler_is_going_out_of_memory._Why_does_DataImportHandler_bring_everything_to_memory.3F >> >>> < >> http://wiki.apache.org/solr/DataImportHandlerFaq?highlight=%28mysql%29%7C%28batchsize%29#I.27m_using_DataImportHandler_with_a_MySQL_database._My_table_is_huge_and_DataImportHandler_is_going_out_of_memory._Why_does_DataImportHandler_bring_everything_to_memory.3F >> > >> >>> >> >>> >> >>> Thanks, >> >>> Shawn >> >>> >> >>> >> > >>
