Hi fellow developers,
We are using "BinFileDataSource" datasource in our DIH config file to index
local file system files. It is able to index the files however, during the next
cycle of indexing, files that were removed from source file system folder is
not removed from index. I believe Solr currently has no capability of doing
this. Can someone please confirm based on your experience?
Also, does delta import work for this datasource? It dosen't seem to work for
me.
<dataSource type="BinFileDataSource" name=" testfilesystem"/>
<entity name="filesystementity" dataSource="testfilesystem" rootEntity="false"
processor="FileListEntityProcessor"
transformer="script:AddGroup"
baseDir="C:\Users\12345\Downloads\Solr\test\"
fileName=".*."
onError="continue" recursive="true"
newerThan="'NOW-365DAYS'">
<field column="fileAbsolutePath" name="id" />
<field column="fileLastModified"
name="lastupdttimestamp" />
<field column="file" name="filename"/>
<entity name="documentImport"
processor="TikaEntityProcessor" url="${filesystementity.fileAbsolutePath}"
format="text" onError="continue">
<field column="text" name="doctext"/>
</entity>
</entity>
Thank you,
Aswath NS
