Re: [OSM-dev] MongoDB possibly unsuitable for OSM data
On Fri, Jul 16, 2010 at 3:50 PM, Ian Dees wrote: > On Fri, Jul 16, 2010 at 2:34 PM, Nolan Darilek > wrote: >> Hey, folks. I suppose that I might be wrong about this, and if so then >> I'd love to be, but I thought that I'd share my recent findings here >> either to inspire work in different directions if possible, or to close >> a possibly useless avenue of work if not. >> >> I have a real-time navigation app that stores its data in MongoDB, using >> its geospatial queries. During the course of my work, I routinely found >> that some queries were reliably slow while others were reliably fast. > I was seeing quite the opposite results with smallish (city-sized) bounding > boxes: I was getting very fast responses (much faster than a second or two). Interesting that you are each looking at Mongo. Mongo came up in two of the lightning talks at SotM last weekend. Serge and Andrii should probably jump in and share their findings with you. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] MongoDB possibly unsuitable for OSM data
On Fri, Jul 16, 2010 at 2:34 PM, Nolan Darilek wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Hey, folks. I suppose that I might be wrong about this, and if so then > I'd love to be, but I thought that I'd share my recent findings here > either to inspire work in different directions if possible, or to close > a possibly useless avenue of work if not. > > I have a real-time navigation app that stores its data in MongoDB, using > its geospatial queries. During the course of my work, I routinely found > that some queries were reliably slow while others were reliably fast. > We're talking differences of seconds, some taking 5, some 30, while > others were completed in under a second. Naturally, this is unsuitable > for an app that needs to provide near real-time feedback. I opened an > issue here, including my dump of an import of Texas' OSM nodes: > > http://jira.mongodb.org/browse/SERVER-1392 > > It seems that I'm running up against limitations in MongoDB's > geohash-based mechanism. It's probably perfectly suitable for most > average geospatial-based searches, but not so much for the case of > rendering OSM data in reliably short bursts of time. The issue has been > marked wontfix. > > I'm open to the possibility that I'm missing something, but have long > suspected that the geospatial support wasn't up to something of this > magnitude. I suppose that it might work as a data storage and > replication system, but if you need to get back data quickly then > MongoDB likely isn't a good fit. > > Anyhow, I thought that I'd share, especially as some of us were > discussing use of MongoDB here a few weeks back. > > I was seeing quite the opposite results with smallish (city-sized) bounding boxes: I was getting very fast responses (much faster than a second or two). I was definitely running into limitations of the Python serializer/deserializer before I was running into limitations of Mongo. This was after inserting most of an entire planet dump. However, it does seem like his explanation is valid: geohashing creates buckets and when those buckets are too big they fill up and make for slow queries. The nice thing about geohashing is that you can have arbitrarily-sized buckets, so I always assumed that they were picking the size based on how many points they saw. Maybe not. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] MongoDB possibly unsuitable for OSM data
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hey, folks. I suppose that I might be wrong about this, and if so then I'd love to be, but I thought that I'd share my recent findings here either to inspire work in different directions if possible, or to close a possibly useless avenue of work if not. I have a real-time navigation app that stores its data in MongoDB, using its geospatial queries. During the course of my work, I routinely found that some queries were reliably slow while others were reliably fast. We're talking differences of seconds, some taking 5, some 30, while others were completed in under a second. Naturally, this is unsuitable for an app that needs to provide near real-time feedback. I opened an issue here, including my dump of an import of Texas' OSM nodes: http://jira.mongodb.org/browse/SERVER-1392 It seems that I'm running up against limitations in MongoDB's geohash-based mechanism. It's probably perfectly suitable for most average geospatial-based searches, but not so much for the case of rendering OSM data in reliably short bursts of time. The issue has been marked wontfix. I'm open to the possibility that I'm missing something, but have long suspected that the geospatial support wasn't up to something of this magnitude. I suppose that it might work as a data storage and replication system, but if you need to get back data quickly then MongoDB likely isn't a good fit. Anyhow, I thought that I'd share, especially as some of us were discussing use of MongoDB here a few weeks back. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkxAtGIACgkQIaMjFWMehWLAdgCfXnCInnxG3BPc1H7y32jhBbwY czcAn2W3g/o5HHp7w9VI6Zlfm2Fa/q0k =PCgC -END PGP SIGNATURE- ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
