[
https://issues.apache.org/jira/browse/LUCENE-7745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16527334#comment-16527334
]
Ishan Chattopadhyaya commented on LUCENE-7745:
----------------------------------------------
Ah, I think I wasn't clear on my intentions behind those numbers.
bq. if it brings any performance - I doubt that, because the call overhead
between Java and CUDA is way too high - in contrast to Postgres where all in
plain C/C++
I wanted to start with those experiments just to prove to myself that there are
no significant overheads or bottlenecks (as we've feared in the past) and that
there can be clear benefits to be realized.
I wanted to try bulk scoring, and chose the distance calculation and sorting as
an example because (1) it leverages two fields, (2) it was fairly isolated &
easy to try out.
In practical usecases of spatial search, the spatial filtering doesn't require
score calculation & sorting on the entire dataset (just those documents that
are in the vicinity of the user point, filtered down by the geohash or bkd tree
node); so in some sense I was trying out an absolute worst case of Lucene
spatial search.
Now, that I'm convinced that this overall approach works and overheads are low,
I can now move on to looking at Lucene internals, maybe starting with scoring
in general (BooleanScorer, for example). Other parts of Lucene/Solr that might
see benefit could be streaming expressions (since they seem computation heavy),
LTR re-ranking etc.
Actually incorporating all these benefits into Lucene would require
considerable effort, and we can open subsequent JIRAs once we've had a chance
to explore them separately. Till then, I'm inclined to keep this issue as a
kitchen sink for all-things-GPU, if that makes sense?
> Explore GPU acceleration for spatial search
> -------------------------------------------
>
> Key: LUCENE-7745
> URL: https://issues.apache.org/jira/browse/LUCENE-7745
> Project: Lucene - Core
> Issue Type: Improvement
> Components: modules/spatial-extras
> Reporter: Ishan Chattopadhyaya
> Assignee: Ishan Chattopadhyaya
> Priority: Major
> Labels: gsoc2017, mentor
> Attachments: gpu-benchmarks.png
>
>
> There are parts of Lucene that can potentially be speeded up if computations
> were to be offloaded from CPU to the GPU(s). With commodity GPUs having as
> high as 12GB of high bandwidth RAM, we might be able to leverage GPUs to
> speed parts of Lucene (indexing, search).
> First that comes to mind is spatial filtering, which is traditionally known
> to be a good candidate for GPU based speedup (esp. when complex polygons are
> involved). In the past, Mike McCandless has mentioned that "both initial
> indexing and merging are CPU/IO intensive, but they are very amenable to
> soaking up the hardware's concurrency."
> I'm opening this issue as an exploratory task, suitable for a GSoC project. I
> volunteer to mentor any GSoC student willing to work on this this summer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
