[
https://issues.apache.org/jira/browse/SOLR-18664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rahul Goswami updated SOLR-18664:
---------------------------------
Description:
h3. Problem
The graph query parser does a breadth-first traversal: at each hop it reads the
{{to}} field of every collected document and adds those values to the next
hop's {{from IN (...)}} frontier query. There is no way to say "collect this
node, but don't expand it".
A common case is a file-system hierarchy, where every document (file or folder)
has its own id in {{id}} and its parent's id in {{{}parentId{}}}:
{noformat}
q={!graph from=parentId to=id returnRoot=true
traversalFilter=name:prefix*}<root folder query>
{noformat}
Only folders ({{{}docType:folder{}}}) can have children, but every file's
({{{}docType:file{}}}) {{id}} also becomes a term in the next hop's query. Each
such term costs a docValues read, a hash insert and a terms-dictionary seek per
segment, and can never match. Files usually far outnumber folders, so most of
the frontier is wasted work.
* {{traversalFilter=docType:folder}} doesn't help: it restricts which
documents are collected, so files would be dropped from the results.
* The existing leaf-node optimization ({{{}resolveLeafNodes(){}}}) only skips
documents with *no value* in the {{to}} field. Here every document has an
{{{}id{}}}, so it never applies.
h3. Proposal
Add an optional {{lookupFilter}} local param. Documents reached by the
traversal (and matching {{{}traversalFilter{}}}) are always collected, but only
those that also match {{lookupFilter}} have their edges followed. It applies at
every level, including the root. When absent, behaviour is unchanged.
{noformat}
{!graph from=parentId to=id traversalFilter=name:prefix*
lookupFilter=docType:folder}<root query>
{noformat}
h3. Back-compat
New optional param; no change when it is absent.
was:
h3. Problem
The graph query parser does a breadth-first traversal: at each hop it reads the
{{to}} field of every collected document and adds those values to the next
hop's {{from IN (...)}} frontier query. There is no way to say "collect this
node, but don't expand it".
A common case is a file-system hierarchy, where every document (file or folder)
has its own id in {{id}} and its parent's id in {{{}parentId{}}}:
{noformat}
q={!graph from=parentId to=id returnRoot=true
traversalFilter=name:prefix*}<root folder query>
{noformat}
Only folders ({{{}docType:folder{}}}) can have children, but every file's
({{{}docType:file{}}}) {{id}} also becomes a term in the next hop's query. Each
such term costs a docValues read, a hash insert and a terms-dictionary seek per
segment, and can never match. Files usually far outnumber folders, so most of
the frontier is wasted work.
* {{traversalFilter=docType:folder}} doesn't help: it restricts which
documents are collected, so files would be dropped from the results.
* The existing leaf-node optimization ({{{}resolveLeafNodes(){}}}) only skips
documents with *no value* in the {{to}} field. Here every document has an
{{{}id{}}}, so it never applies.
h3. Proposal
Add an optional {{lookupFilter}} local param. Documents reached by the
traversal (and matching {{{}traversalFilter{}}}) are always collected, but only
those that also match {{lookupFilter}} have their edges followed. It applies at
every level, including the root. When absent, behaviour is unchanged.
{noformat}
{!graph from=parentId to=id traversalFilter=name:prefix*
lookupFilter=docType:folder}<root query>
{noformat}
h3. Back-compat
New optional param; no change when it is absent.
> Add lookupFilter to the graph query parser to skip edge lookups for nodes
> that can't have children
> --------------------------------------------------------------------------------------------------
>
> Key: SOLR-18664
> URL: https://issues.apache.org/jira/browse/SOLR-18664
> Project: Solr
> Issue Type: Improvement
> Components: query parsers
> Reporter: Rahul Goswami
> Assignee: Rahul Goswami
> Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> h3. Problem
> The graph query parser does a breadth-first traversal: at each hop it reads
> the {{to}} field of every collected document and adds those values to the
> next hop's {{from IN (...)}} frontier query. There is no way to say "collect
> this node, but don't expand it".
> A common case is a file-system hierarchy, where every document (file or
> folder) has its own id in {{id}} and its parent's id in {{{}parentId{}}}:
> {noformat}
> q={!graph from=parentId to=id returnRoot=true
> traversalFilter=name:prefix*}<root folder query>
> {noformat}
> Only folders ({{{}docType:folder{}}}) can have children, but every file's
> ({{{}docType:file{}}}) {{id}} also becomes a term in the next hop's query.
> Each such term costs a docValues read, a hash insert and a terms-dictionary
> seek per segment, and can never match. Files usually far outnumber folders,
> so most of the frontier is wasted work.
> * {{traversalFilter=docType:folder}} doesn't help: it restricts which
> documents are collected, so files would be dropped from the results.
> * The existing leaf-node optimization ({{{}resolveLeafNodes(){}}}) only
> skips documents with *no value* in the {{to}} field. Here every document has
> an {{{}id{}}}, so it never applies.
> h3. Proposal
> Add an optional {{lookupFilter}} local param. Documents reached by the
> traversal (and matching {{{}traversalFilter{}}}) are always collected, but
> only those that also match {{lookupFilter}} have their edges followed. It
> applies at every level, including the root. When absent, behaviour is
> unchanged.
> {noformat}
> {!graph from=parentId to=id traversalFilter=name:prefix*
> lookupFilter=docType:folder}<root query>
> {noformat}
> h3. Back-compat
> New optional param; no change when it is absent.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]