> I am rather new to Solr and have a question.
>
> We have around 200.000 txt files which are placed into the
> file cloud.
> The file path is something similar to this:
>
> file/97/8f/840/fa4-1.txt
> file/a6/9d/ab0/ca2-2.txt etc.
>
> and we also store the metadata (like title, description,
> tags etc)
> about these files in the mysql server. So, what I want to
> do is to
> index title, description, tags and other data from mysql,
> and also get
> the txt file from file server, and link them as one record
> for
> searching, but I could not figure out how to automatize
> this process.
> I can give the path from the sql query like, Select id,
> title,
> description, file_path, and then solr can use this path to
> retrieve
> txt file, but I don't know whether is it possible or not.
I haven't done it by myself but something like/similar this should do it:
<dataConfig>
<dataSource type="JdbcDataSource" name="ds-db" driver="com.mysql.jdbc.Driver"
url="jdbc:mysql://" user="user" password="pass" batchSize="-1" readOnly="true"/>
<dataSource type="FileDataSource" name="ds-file" encoding="UTF-8" />
<document name="docs">
<entity name="f" dataSource="ds-db" pk="id" query="Select id, title,
description, file_path" >
<field column="title" name="title"/>
<field column="description" name="description"/>
<entity processor="PlainTextEntityProcessor" name="x" url="${f.file_path}"
dataSource="ds-file">
<!-- copies the text to a field called 'text' in Solr-->
<field column="plainText" name="text"/>
</entity>
</entity>
</document>
</dataConfig>
