On Wed, Jan 12, 2011 at 8:49 PM, alexei <achugu...@gmail.com> wrote:
[...]
> Unfortunately reorganizing the data is not an option for me.
> Multiple databases exist and a third party is taking care of
> populating them. Once a database reaches a certain size, a switch
> occurs and a new database is created with the same table structure.
OK, I understand.
> Gora Mohanty-3 wrote:
>>
>> I meant a script that runs the query that defines the datasources for all
>> fields, writes a Solr DIH configuration file, and then initiates a
>> dataimport.
>>
> Ok, so the query would select only the articles for which the data is
> sitting in a specific datasource. Then, only that one datasource would be
> indexed.
> For each additional datasource would the script initiate another full-import
> with the clean attribute set to false?
I do not think that I am completely understanding your use case.
Would it be possible for you to describe it in detail? Here is my
current view of it:
* From some SELECT statement, it is possible for you to tell
which datasource what field should come from in the next import.
* If so, before the start of a data import, a script can run that same
SELECT statement, and figure out what belongs where.
* In that case, the script can do the following:
- Write a DIH configuration file from its knowledge of where the
fields in the next import are coming from.
- Do a reload-config to get the new DIH configuration.
- Initiate a data import
* It is not clear to me how a delta import, and similar things fit
into this scenario. I.e., are you also going to be dealing with
updates of documents that already exist in the Solr index?
However, we can cross that bridge when we come to it.
> I tried to make some changes to DIH that comes with Solr 1.4.1
> The getResolvedEntityAttribute("dataSource"); method seems to so the trick.
> Here is the modified code. It feels awkward but it seems to work.
[...]
> I hope I am not breaking any other functionality...
> Would it be possible to add something like this to a future release?
I am sorry. As things stand, while I do want to be able to get the
time to become a contributor to Solr code, it is beyond my current
understanding of it to be able to comment on the above. I think that
you have the right idea, but am unable to say for sure. Maybe someone
more well-versed in Solr can chip in. I would definitely recommend
that you open a JIRA ticket, and attach this patch. That way, at least
it remains on record. Please include a description of your use case
in the ticket.
Regards,
Gpra
