Hmm, I just checked out your code. Out of curiosity, what hardware did
you run it on where importing a planet took several days?
It looks like our schemas are mostly identical, but in my experience,
MongoDB used more and more time importing the index as the import
continued. An import of the dataset for TX took several hours, but
import speeds dropped off markably as the import continued and,
presumably, as the geoindex grew.
I wondered if perhaps MongoDB's geospatial support wasn't a good fit for
mapping apps, especially as I experienced some very long query times for
no reason I could determine, but now I'm wondering if the slow imports
at least might be due to some hidden detail in my own implementation.
Also, how much disk space did the final import use?
On 07/03/2010 03:43 PM, Ian Dees wrote:
On Sat, Jul 3, 2010 at 1:17 PM, Nolan Darilek <[email protected]
<mailto:[email protected]>> wrote:
On 07/03/2010 01:09 PM, Nolan Darilek wrote:
On 07/02/2010 01:52 PM, Serge Wroclawski wrote:
Similarly, Ian Dees and I have written a server using
MongoDB, which
also provides functionality such as auto-sharding and built in
map/reduce.
Is this work available anywhere? How did you find performance
to be, and to what uses did you put it?
I've done some experiments creating a LibOSM MongoDB backend
and found its performance fairly bad, but I don't have the
most optimal server for it, and probably didn't use MongoDB to
its limits. If you experienced good performance for real-time
operations then I'd be very interested in seeing how you
managed it so I might adopt the techniques and see if I have
any better luck.
It seemed to me that a dump of the entire planet would require
a substantial server to serve up, so I abandoned the work, but
would very much like to revive it if it's at all workable.
The code is here: http://github.com/iandees/mongosm
It took several days to import a planet file. The majority of the CPU
time was spent serializing/deserializing BSON in Python and the Mongo
server had very little CPU time so if I use a language with a faster
BSON implementation it might be faster.
Serge was working on a way to import diff files to maintain mintutely
updates.
I was working on an HTTP API interface.
_______________________________________________
dev mailing list
[email protected]
http://lists.openstreetmap.org/listinfo/dev
