Hi David,
First of all, thanks for the development of lucene - it is an amazing and
unique library.
Sorry, 10x was a very rough estimation - lucene is actually 4 times slower.
When using Lucene, I can perform around 700 queries/second (that's 8
threads on 8 core machine macbook Pro with ssd disks). With JTS STRTree I
was able to get around 2800 queries/sec, so that's around 4x slowdown. I
was counting only query performance, not indexing.
I am storing BB rectangles as geometry and the real geometry in WKB format
as the value field of the record. And I am using QuadPrefixTree.
One thing I have noticed is that Lucene is dealing with lat/lng coordinates
only - therefore it wont allow any other reference systems (I am using
British reference grid: http://spatialreference.org/ref/epsg/27700/ ), so I
had to scale down all bounding boxes so the coordinates fit into 0-180
interval.
I haven't tried any of the standalone databases as I believe the simple
network overhead will kill all possible performance benefits. Also for
other reasons I do not want to deal with those.
I still believe the operations I am performing on the RTree are relatively
simple and Lucene is optimised for much more general use, so I have some
hopes to enhance it's performance.
D.
On Fri, Jan 17, 2014 at 3:39 PM, Smiley, David W. <[email protected]> wrote:
> Whoops; forgot to reply-all.
>
>
> From: <Smiley>, "Smiley, David W." <[email protected]>
> Date: Friday, January 17, 2014 at 10:15 AM
> To: Demeter Sztanko <[email protected]>
> Subject: Re: [Jts-topo-suite-user] Persistent STR tree
>
> Thanks for sharing your experience with Lucene-spatial. I’m
> responsible for a large part of it. I don’t think you’re ever going to get
> the performance of an in-memory structure to compare to an on-disk one
> (even SSD). Of course if you find one then let me know. FWIW I’m looking
> to improve the accuracy & performance of lucene-spatial a lot this year.
> Can you tell me if the indexed spatial objects are all points or if it’s
> mostly non-points? And was the 10x slower just query performance or did
> that include indexing?
>
> In the NoSQL space (or shall we say… not a relational database space),
> the systems with the best spatial support to my knowledge are MongoDB,
> CouchDB (spatial module is add-on separately), and Lucene-spatial. Your
> data set isn’t huge though; I’d try PostGIS if I were you. And I’m very
> impressed with what I see in SQL Server.
>
> Good luck,
> ~ David Smiley
>
> From: Demeter Sztanko <[email protected]>
> Date: Friday, January 17, 2014 at 7:56 AM
> To: "[email protected]" <
> [email protected]>
> Subject: [Jts-topo-suite-user] Persistent STR tree
>
> Hi all,
>
> I need to store around 50M objects in a spatial index (I need only
> support for bulk insert and concurrent intersection() operations). I need
> then to semi-randomly access the objects (that is, I probably will have 300
> requests within one location, then another 300 in another random location,
> etc.)
>
> STRTree is great and fast, however I need around 50GB of RAM for fitting
> the tree which is unfortunately too expensive for me to maintain in long
> term.
>
> I need a solution that can run on 1Gb of RAM and SSD disks (it's a
> digitalocean cloud instance)
>
> I have also tried using Lucene for storing spatial index, which is also
> feasible but around 10 times slower even on SSD disks.
>
> I was wondering if you know of any other minimal java libraries that can
> do what I am looking for yet they are still relatively fast.
>
>
> Thanks,
>
> D.
>
>
>
> ------------------------------------------------------------------------------
> CenturyLink Cloud: The Leader in Enterprise Cloud Services.
> Learn Why More Businesses Are Choosing CenturyLink Cloud For
> Critical Workloads, Development Environments & Everything In Between.
> Get a Quote or Start a Free Trial Today.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
> _______________________________________________
> Jts-topo-suite-user mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/jts-topo-suite-user
>
>
------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________
Jts-topo-suite-user mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/jts-topo-suite-user
