[
https://issues.apache.org/jira/browse/SOLR-1715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14309706#comment-14309706
]
Toke Eskildsen commented on SOLR-1715:
--------------------------------------
We need something like this for defining subsets of our net archive index
(billions of documents), so I am considering writing a patch. My idea for a
solution is as follows:
In a request, a user can state a key and a URL. These two values are stored for
the lifetime of the Solr JVM (or until updated/deleted). When a user requests a
filter for the given key, it is evaluated to a filter docset, which is stored
until the index is updated.
Evaluation of dhe content from the URL is processed in a streaming manner,
without any size limit. Each line is treated as a Solr query and the result ing
docIDs are enabled in the docset. Treating each line as a query means that
"everything" can be expressed, from corpus spanning {{*:*}} through large
{{TermsQueries}} to a billion {{ID:xxxxxx statements}}. The downside is size
(and as part of that speed), so other types of streams (bitmap with the content
for the docset, list of ID-values, etc) might be worth adding at a later point.
> Externally Generated Filters
> ----------------------------
>
> Key: SOLR-1715
> URL: https://issues.apache.org/jira/browse/SOLR-1715
> Project: Solr
> Issue Type: New Feature
> Components: search
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 4.9, Trunk
>
>
> I'd like to have a way of loading document filters from externally generated
> sources. For instance, I may have run a clustering task over my corpus and
> the results may be stored in HDFS (Hadoop). A given cluster will have a
> centroid and a bunch of points that are in the cluster. The points are
> labeled by doc id (the Solr doc id) from the index.
> Another example is I may have an external security system that can provide
> info on which documents are available to a user.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
