What about rewriting this stuff in C? I have written a MySQL importer in
C some time ago, so what about reusing its XML parsing part?
If you are interested, I'll put it on Github.
Andi
Am 03.07.10 22:43, schrieb Ian Dees:
On Sat, Jul 3, 2010 at 1:17 PM, Nolan Darilek <[email protected]
<mailto:[email protected]>> wrote:
On 07/03/2010 01:09 PM, Nolan Darilek wrote:
On 07/02/2010 01:52 PM, Serge Wroclawski wrote:
Similarly, Ian Dees and I have written a server using
MongoDB, which
also provides functionality such as auto-sharding and built in
map/reduce.
Is this work available anywhere? How did you find performance
to be, and to what uses did you put it?
I've done some experiments creating a LibOSM MongoDB backend
and found its performance fairly bad, but I don't have the
most optimal server for it, and probably didn't use MongoDB to
its limits. If you experienced good performance for real-time
operations then I'd be very interested in seeing how you
managed it so I might adopt the techniques and see if I have
any better luck.
It seemed to me that a dump of the entire planet would require
a substantial server to serve up, so I abandoned the work, but
would very much like to revive it if it's at all workable.
The code is here: http://github.com/iandees/mongosm
It took several days to import a planet file. The majority of the CPU
time was spent serializing/deserializing BSON in Python and the Mongo
server had very little CPU time so if I use a language with a faster
BSON implementation it might be faster.
Serge was working on a way to import diff files to maintain mintutely
updates.
I was working on an HTTP API interface.
_______________________________________________
dev mailing list
[email protected]
http://lists.openstreetmap.org/listinfo/dev
_______________________________________________
dev mailing list
[email protected]
http://lists.openstreetmap.org/listinfo/dev
